Add standalone WebRTC VAD and harden detector lifecycle

docs/guide/build.md

@@ -26,6 +26,17 @@ On the first run, the WebRTC source tree will be loaded into the `/<user home>/w
 | webrtc.src.dir     | The absolute checkout path for the WebRTC source tree. | /\<user_home\>/webrtc       |
 | webrtc.install.dir | The install path for the compiled WebRTC library. Is also used to link against a pre-compiled WebRTC library to reduce build time. | /\<user_home\>/webrtc/build |
 
+## WebRTC patches and cached builds
+
+The native build applies the patches in
+`webrtc-jni/src/main/cpp/dependencies/webrtc/patches/common/` on every platform,
+followed by the platform-specific patches. Common safety patches are required:
+configuration stops if one cannot be applied.
+
+A compiled WebRTC build records its GN arguments, extra targets, and patch
+fingerprint. Changes to patch content invalidate both build-tree and installed
+caches, so a native rebuild also includes the updated patches.
+
 [build-linux-ubuntu]: https://chromium.googlesource.com/chromium/src/+/master/docs/linux/build_instructions.md#system-requirements
 [build-linux-other]: https://chromium.googlesource.com/chromium/src/+/master/docs/linux/build_instructions.md#Notes-for-other-distros
 [build-windows]: https://chromium.googlesource.com/chromium/src/+/master/docs/windows_build_instructions.md#visual-studio

docs/tools/audio/voice-activity-detector.md

@@ -1,104 +1,153 @@
-# Voice Activity Detector
+# Voice Activity Detection
 
-The Voice Activity Detector (VAD) helps you determine when speech is present in an audio stream. It analyzes short chunks of PCM audio and returns the probability that the chunk contains voice.
+Two WebRTC detectors are available. Choose according to the result your
+application needs; neither is guaranteed to classify every recording better.
+Both analyze mono signed 16-bit PCM and maintain state across audio frames.
 
-This can be used to:
-- Drive push-to-talk or auto-mute logic
-- Skip encoding/sending silence to save bandwidth
-- Trigger UI indicators when the user is speaking
+| API | Native implementation | Result | Input frames |
+| --- | --- | --- | --- |
+| `VoiceActivityDetector` | Combined standalone and pitch-based detector | Estimated probability from 0 to 1 | 10 ms |
+| `StandaloneVoiceActivityDetector` | Standalone WebRTC VAD directly | Boolean speech decision | 10, 20 or 30 ms |
 
-API: `dev.onvoid.webrtc.media.audio.VoiceActivityDetector`
+The existing `VoiceActivityDetector.process()` and `getLastVoiceProbability()`
+methods remain available with the original probability estimator. No migration
+is required. The standalone detector is an additional class, with its own state.
 
-## Overview
+## Probability estimator
 
-`VoiceActivityDetector` exposes a minimal API:
-- `process(byte[] audio, int samplesPerChannel, int sampleRate)`: Analyze one audio frame.
-- `getLastVoiceProbability()`: Retrieve the probability (0.0..1.0) that the last processed frame contained voice.
-- `dispose()`: Release native resources. Always call this when done.
-
-Internally, VAD uses a native implementation optimized for real-time analysis. The class itself does not perform resampling or channel mixing, so provide audio matching the given `sampleRate` and expected format.
-
-## Audio format expectations
-
-- PCM signed 16-bit little-endian (typical Java byte[] from microphone capture via this library)
-- Mono is recommended. If you have stereo, downmix to mono before calling `process` or pass samples-per-channel accordingly
-- Frame size: commonly 10 ms per call (e.g., 160 samples at 16 kHz for 10 ms)
-- Supported sample rates: 8 kHz, 16 kHz, 32 kHz, 48 kHz (use one of these for best results)
-
-## Basic usage
+`dev.onvoid.webrtc.media.audio.VoiceActivityDetector` combines the standalone VAD
+with pitch and spectral features. Your application chooses a threshold on the
+returned probability. The result is delayed by a few chunks and is not updated
+on every call. WebRTC initially returns `1.0` until enough audio has arrived;
+do not interpret that initial value as observed speech.
 
 ```java
 import dev.onvoid.webrtc.media.audio.VoiceActivityDetector;
 
-// Create the detector
 VoiceActivityDetector vad = new VoiceActivityDetector();
-
 try {
-    // Example parameters
-    int sampleRate = 16000;                 // 16 kHz
-    int frameMs = 10;                       // 10 ms frames
-    int samplesPerChannel = sampleRate * frameMs / 1000; // 160 samples
+    byte[] frame = new byte[320]; // 10 ms, mono signed 16-bit PCM at 16 kHz
+    // In your audio loop, fill frame from the source, then:
+    vad.process(frame, 160, 16000);
+    float probability = vad.getLastVoiceProbability();
+    // After the initial buffering period, apply your application's threshold.
+    boolean speech = probability > 0.7f; // Example threshold, not a universal setting.
+}
+finally {
+    vad.dispose();
+}
+```
 
-    // audioFrame must contain 16-bit PCM data for one frame (mono)
-    byte[] audioFrame = new byte[samplesPerChannel * 2]; // 2 bytes per sample
+Pass exactly `sampleRate / 100` samples per call. The byte buffer must contain at
+least twice that many bytes; trailing bytes are ignored. Samples are mono signed
+16-bit PCM in native byte order, matching the library's audio callbacks. Supported
+sample rates are 8000, 16000, 32000 and 48000 Hz; this detector resamples to 16 kHz
+internally. Downmix stereo before calling `process()`.
 
-    // Fill audioFrame from your audio source here
-    // ...
+Keep one instance per continuous stream and create a new instance when beginning
+a new stream or changing sample rates. The standalone detector's `Mode` setting
+does not configure this estimator. Its probability is a model estimate, not a
+guarantee that a given percentage of frames will be classified correctly.
 
-    // Analyze frame
-    vad.process(audioFrame, samplesPerChannel, sampleRate);
+## Standalone detector
 
-    // Query probability of voice in the last frame
-    float prob = vad.getLastVoiceProbability(); // 0.0 .. 1.0
+`dev.onvoid.webrtc.media.audio.StandaloneVoiceActivityDetector` provides direct
+boolean decisions and four aggressiveness modes. It does not expose a probability
+score, perform channel mixing, or resample input.
 
-    boolean isSpeaking = prob >= 0.5f; // choose a threshold that works for your app
+```java
+import dev.onvoid.webrtc.media.audio.StandaloneVoiceActivityDetector;
 
+StandaloneVoiceActivityDetector vad = new StandaloneVoiceActivityDetector(
+        StandaloneVoiceActivityDetector.Mode.AGGRESSIVE);
+try {
+    short[] frame = new short[160]; // 10 ms of mono audio at 16 kHz
+    // Fill frame with PCM samples from your audio source.
+    boolean speech = vad.isSpeech(frame, 16000);
 }
 finally {
-    // Always release resources
     vad.dispose();
 }
 ```
 
-## Continuous processing loop
+Provide one mono PCM frame in a `short[]` with exactly one of these lengths:
+
+| Sample rate | 10 ms | 20 ms | 30 ms |
+| --- | --- | --- | --- |
+| 8000 Hz | 80 | 160 | 240 |
+| 16000 Hz | 160 | 320 | 480 |
+| 32000 Hz | 320 | 640 | 960 |
+| 48000 Hz | 480 | 960 | 1440 |
+
+Lengths are sample counts, not bytes. Numeric `short` values have no byte order;
+when decoding a PCM `byte[]`, use the source's byte order.
+
+Modes are `QUALITY` (the default), `LOW_BITRATE`, `AGGRESSIVE`, and
+`VERY_AGGRESSIVE`. Higher aggressiveness rejects more noise and can miss more
+speech. `setMode(mode)` changes aggressiveness without clearing history.
+
+Keep one detector per independent stream and feed frames in order. Do not reset
+between frames: the detector uses adaptive state and built-in hangover, so a few
+frames after speech ends can still be classified as speech. Call `reset()` when
+starting a new stream or changing sample rates; it preserves the selected mode.
+
+Downmix stereo and resample unsupported rates before detection. The
+[Audio Converter](/tools/audio/audio-converter) accepts 10 ms frames and can do
+both operations. Split larger callbacks or buffer partial callbacks into complete
+frames.
+
+## Lifecycle and concurrency
+
+Both constructors load the platform native library. A `PeerConnectionFactory` is
+not required. Include the matching native classifier jar as usual.
+
+Calls, including `dispose()`, are serialized on each detector. Callers must still
+preserve stream order and avoid modifying an input array during detection.
+Disposal is idempotent. Processing or querying a disposed detector throws
+`IllegalStateException`, as do standalone configuration/reset calls after disposal.
+Invalid frame lengths/rates throw `IllegalArgumentException`; null input buffers
+or standalone modes throw `NullPointerException`.
+
+The standalone detector reports failure to allocate its native instance as
+`OutOfMemoryError`. The probability estimator retains WebRTC's native
+allocation-failure behavior; catching Java exceptions does not guarantee recovery
+from native memory exhaustion.
+
+A probability detector and a standalone detector do not share history, even when
+they receive the same audio. Feeding, configuring, resetting, or disposing one
+does not update the other. Neither API automatically changes encoding, DTX, mute
+state, or transmitted audio.
+
+## Audio track integration
+
+The examples module includes `VoiceActivityExample`, an `AudioTrackSink` adapter
+using `StandaloneVoiceActivityDetector`. It buffers callback data into 10 ms
+frames, converts mono or stereo input to 16 kHz mono with `AudioConverter`, and
+reports each boolean decision. It also supports 44.1 kHz input.
 
 ```java
-VoiceActivityDetector vad = new VoiceActivityDetector();
+import dev.onvoid.webrtc.examples.VoiceActivityExample;
 
+VoiceActivityExample sink = new VoiceActivityExample(speech -> {
+    // Enqueue the decision for your application/UI; return promptly.
+});
+track.addSink(sink);
 try {
-    int sampleRate = 16000;
-    int frameMs = 10;
-    int samplesPerChannel = sampleRate * frameMs / 1000; // 160 samples
-    byte[] audioFrame = new byte[samplesPerChannel * 2];
-
-    while (running) {
-        // Read PCM frame from your capture pipeline into audioFrame
-        // ...
-
-        vad.process(audioFrame, samplesPerChannel, sampleRate);
-        float prob = vad.getLastVoiceProbability();
-
-        if (prob > 0.8f) {
-            // High confidence of speech
-            // e.g., enable VU meter, unmute, or mark active speaker
-        }
-        else {
-            // Likely silence or noise
-        }
-    }
+    // Keep the sink attached while receiving audio.
 }
 finally {
-    vad.dispose();
+    track.removeSink(sink);
+    sink.close();
 }
 ```
 
-## Tips and best practices
-
-- Threshold selection: Start with 0.5–0.8 and tune for your environment.
-- Frame size consistency: Use a consistent frame duration and sample rate.
-- Resource management: VAD holds native resources; ensure `dispose()` is called.
-- Preprocessing: Consider using `AudioProcessing` (noise suppression, gain control) before VAD for improved robustness in noisy environments. See the Audio Processing guide.
+The adapter invokes the consumer on the audio callback thread. Keep work there
+short. Apply application-level timing to produce stable speech-start/speech-end
+events, rather than toggling a UI or recording for every frame. Audio format
+changes clear the adapter's partial frame and reset its detector.
 
 ## Related guides
 
-- [Audio Processing](/guide/audio/audio-processing)
+- [Audio Processing](/guide/audio/audio-processing)
+- [Audio Converter](/tools/audio/audio-converter)
+- [Audio Recorder](/tools/audio/audio-recorder)

webrtc-examples/src/main/java/dev/onvoid/webrtc/examples/VoiceActivityExample.java

@@ -0,0 +1,101 @@
+/*
+ * Copyright 2026 Alex Andres
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package dev.onvoid.webrtc.examples;
+
+import java.nio.ByteBuffer;
+import java.nio.ByteOrder;
+import java.util.function.Consumer;
+import java.util.Objects;
+
+import dev.onvoid.webrtc.media.audio.AudioConverter;
+import dev.onvoid.webrtc.media.audio.AudioTrackSink;
+import dev.onvoid.webrtc.media.audio.StandaloneVoiceActivityDetector;
+
+/**
+ * Attach to an AudioTrack with addSink(), then removeSink() before close().
+ * Buffers arbitrary callback lengths into 10 ms frames and converts interleaved
+ * signed 16-bit PCM to 16 kHz mono, including input at 44.1 kHz. The consumer runs
+ * on the audio callback thread and should return promptly. It receives raw frame
+ * decisions; a UI should apply its own speaking-state timing off this thread.
+ */
+public final class VoiceActivityExample implements AudioTrackSink, AutoCloseable {
+
+    private final Consumer<Boolean> onDecision;
+    private final StandaloneVoiceActivityDetector detector;
+    private AudioConverter converter;
+    private int rate;
+    private int channels;
+    private byte[] pending;
+    private int buffered;
+    private final byte[] converted = new byte[320];
+    private final short[] mono = new short[160];
+    private final ByteBuffer convertedSamples = ByteBuffer.wrap(converted).order(ByteOrder.nativeOrder());
+    private boolean closed;
+
+    public VoiceActivityExample(Consumer<Boolean> onDecision) {
+        this.onDecision = Objects.requireNonNull(onDecision, "onDecision");
+        detector = new StandaloneVoiceActivityDetector(StandaloneVoiceActivityDetector.Mode.AGGRESSIVE);
+    }
+
+    @Override
+    public synchronized void onData(byte[] data, int bitsPerSample, int sampleRate, int channels, int frames) {
+        if (closed) {
+            return;
+        }
+        if (bitsPerSample != 16 || sampleRate < 8000 || sampleRate > 96000 || sampleRate % 100 != 0
+                || (channels != 1 && channels != 2) || frames < 0
+                || (long) frames * channels * 2 != data.length) {
+            throw new IllegalArgumentException("Expected interleaved 16-bit mono/stereo PCM at a whole 10 ms rate");
+        }
+        if (rate != sampleRate || this.channels != channels) {
+            AudioConverter replacement = new AudioConverter(sampleRate, channels, 16000, 1);
+            if (converter != null) {
+                converter.dispose();
+            }
+            converter = replacement;
+            rate = sampleRate;
+            this.channels = channels;
+            pending = new byte[sampleRate / 100 * channels * 2];
+            buffered = 0;
+            detector.reset();
+        }
+        int offset = 0;
+        while (offset < data.length && !closed) {
+            int count = Math.min(data.length - offset, pending.length - buffered);
+            System.arraycopy(data, offset, pending, buffered, count);
+            buffered += count;
+            offset += count;
+            if (buffered == pending.length) {
+                converter.convert(pending, converted);
+                convertedSamples.asShortBuffer().get(mono);
+                buffered = 0;
+                onDecision.accept(detector.isSpeech(mono, 16000));
+            }
+        }
+    }
+
+    @Override
+    public synchronized void close() {
+        if (!closed) {
+            closed = true;
+            if (converter != null) {
+                converter.dispose();
+            }
+            detector.dispose();
+        }
+    }
+}

webrtc-jni/src/main/cpp/dependencies/jni-voithos/include/JavaUtils.h

@@ -42,13 +42,18 @@
 		}																				\
 	} while(0)
 
+// Throws the requested Java exception without replacing an already pending one.
+// Uses JNI directly, so reporting an allocation failure needs no C++ allocation.
+void ThrowJavaException(JNIEnv * env, const char * className, const char * message) noexcept;
 void ThrowCxxJavaException(JNIEnv * env);
 JNIEnv * AttachCurrentThread();
 bool ExceptionCheck(JNIEnv * env);
 jclass FindClass(JNIEnv * env, const char * name);
 jmethodID GetMethod(JNIEnv * env, jclass cls, const char * name, const char * sig);
 jmethodID GetStaticMethod(JNIEnv * env, jclass cls, const char * name, const char * sig);
-jfieldID GetHandleField(JNIEnv * env, jobject obj, const std::string & fieldName);
+// Specify the declaring class when subclasses may hide a handle field.
+jfieldID GetHandleField(JNIEnv * env, jobject obj, const std::string & fieldName,
+	const char * declaringClass = nullptr);
 jfieldID GetFieldID(JNIEnv * env, jobject obj, const std::string & fieldName, const char * type);
 jfieldID GetFieldID(JNIEnv * env, jclass cls, const std::string & fieldName, const char * type);
 
@@ -78,9 +83,10 @@ jlong GetHandleLong(JNIEnv * env, jobject obj)
 }
 
 template<typename T>
-T * GetHandle(JNIEnv * env, jobject obj, const std::string & handleName)
+T * GetHandle(JNIEnv * env, jobject obj, const std::string & handleName,
+	const char * declaringClass = nullptr)
 {
-	jfieldID field = GetHandleField(env, obj, handleName);
+	jfieldID field = GetHandleField(env, obj, handleName, declaringClass);
 
 	if (!field) {
 		ExceptionCheck(env);
@@ -97,9 +103,10 @@ T * GetHandle(JNIEnv * env, jobject obj)
 }
 
 template<typename T>
-void SetHandle(JNIEnv * env, jobject obj, const std::string & handleName, T * t)
+void SetHandle(JNIEnv * env, jobject obj, const std::string & handleName, T * t,
+	const char * declaringClass = nullptr)
 {
-	jfieldID field = GetHandleField(env, obj, handleName);
+	jfieldID field = GetHandleField(env, obj, handleName, declaringClass);
 
 	if (!field) {
 		ExceptionCheck(env);