Teleop example

Teleop example

⚠️ Not yet released: in-development v1.5.0-SNAPSHOT capability; interface and protocol may change. Do not use in production before the official release.

New in v1.5.0. End-to-end Java walkthrough of enter → startAudio → appendAudioDelta → doneAudio → exit. The runnable version lives at agentsdk_for_java/example/src/main/java/com/agibot/aiem/teleop/TeleopExample.java.

Minimum example: TeleopExample.java

java
package com.agibot.aiem.teleop;

import com.agibot.aiem.sdk.AgentAuthCallback;
import com.agibot.aiem.sdk.AgentParam;
import com.agibot.aiem.sdk.AgentSdk;
import com.agibot.aiem.sdk.IdGenerator;
import com.agibot.aiem.sdk.teleop.TeleopEnterCallback;
import com.agibot.aiem.sdk.teleop.TeleopExitCallback;
import com.agibot.aiem.sdk.teleop.TeleopKeepaliveCallback;
import com.agibot.aiem.sdk.teleop.TeleopRequest;
import lombok.extern.slf4j.Slf4j;

@Slf4j
public class TeleopExample {

    public static void main(String[] args) throws InterruptedException {

        String url       = "wss://open.agibot.com/api/V1/open-portal/app/wss/agent-sdk";
        String appId     = "<appId from the LinkSoul open platform>";
        String appKey    = "<appKey from the LinkSoul open platform>";
        String appSecret = "<appSecret from the LinkSoul open platform>";
        String agentId   = "<target robot agentId>";

        // 1. Create the SDK instance
        AgentSdk agentSdk = AgentSdk.create(url, appId, appKey, appSecret);

        // 2. Register auth callback
        agentSdk.registerAuth(new AgentAuthCallback() {
            @Override
            public void onAuthState(String appId, int code, String msg) {
                System.out.println("onAuthState => appId: " + appId
                        + ", code: " + code + ", msg: " + msg);
            }
        });

        // 3. Open the WebSocket connection
        agentSdk.initialize();
        Thread.sleep(2000);

        // 4. Register a teleop request
        String teleopId = IdGenerator.generateTeleopId();
        TeleopRequest teleopRequest = new TeleopRequest(agentSdk, agentId, teleopId);
        agentSdk.registerTeleop(teleopRequest);

        // 5. Enter the teleop session
        AgentParam enterParam = AgentParam.create();
        teleopRequest.enter(enterParam, new TeleopEnterCallback() {
            @Override
            public void onEnterAck(String teleopId, String eventId, int code, String msg) {
                log.info("onEnterAck => teleopId {}, eventId {}, code {}, msg {}", teleopId, eventId, code, msg);
            }
        }, 2000);

        // 6. Push one audio segment (fire-and-forget)
        AgentParam startParam = AgentParam.create();
        startParam.setString("role", "male");       // male / female
        startParam.setInteger("threshold", 0);     // -14 – 14
        String eventId = teleopRequest.startAudio(startParam);
        // real code should call appendAudioDelta once per 20-40 ms PCM/Opus chunk (base64-encoded)
        teleopRequest.appendAudioDelta(eventId, "");
        teleopRequest.doneAudio(eventId);

        // 7. Probe robot online (in real code drive this on a 10 s scheduler — see "Periodic keepalive" below)
        teleopRequest.keepalive(new TeleopKeepaliveCallback() {
            @Override
            public void onKeepaliveAck(String teleopId, String eventId, int code, String msg) {
                log.info("onKeepaliveAck => teleopId {}, eventId {}, code {}, msg {}", teleopId, eventId, code, msg);
            }
        }, 2000);

        // 8. Exit the teleop session
        teleopRequest.exit(new TeleopExitCallback() {
            @Override
            public void onExitAck(String teleopId, String eventId, int code, String msg) {
                log.info("onExitAck => teleopId {}, eventId {}, code {}, msg {}", teleopId, eventId, code, msg);
            }
        }, 2000);

        // 9. Unregister and release (omitted here)
        // agentSdk.unregisterTeleop(teleopId);
        // agentSdk.release();
    }
}

Periodic keepalive

After enter succeeds you should probe every 10 seconds. Any onKeepaliveAck with code != 0 means the robot is offline — stop the timer, exit the session, unregister the request so no further audio is wasted on a dead peer.

java
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicBoolean;

ScheduledExecutorService keepaliveScheduler = Executors.newSingleThreadScheduledExecutor(r -> {
    Thread t = new Thread(r, "teleop-keepalive");
    t.setDaemon(true);
    return t;
});
AtomicBoolean robotOnline = new AtomicBoolean(true);

keepaliveScheduler.scheduleAtFixedRate(() -> {
    if (!robotOnline.get()) return;
    teleopRequest.keepalive(new TeleopKeepaliveCallback() {
        @Override
        public void onKeepaliveAck(String teleopId, String eventId, int code, String msg) {
            if (code == 0) return;
            if (!robotOnline.compareAndSet(true, false)) return;
            log.warn("robot offline => code={}, msg={}", code, msg);
            keepaliveScheduler.shutdown();
            teleopRequest.exit(new TeleopExitCallback() {
                @Override
                public void onExitAck(String teleopId, String eventId, int code, String msg) {
                    agentSdk.unregisterTeleop(teleopId);
                }
            }, 2000);
        }
    }, 2000);
}, 10, 10, TimeUnit.SECONDS);

Also keepaliveScheduler.shutdown() when your business logic invokes exit, so probes stop as well.

Common tweaks

GoalWhere to change
Multiple utterances in one sessionCall startAudio(startParam) again for each utterance to get a fresh eventId; reuse that eventId for its appendAudioDelta / doneAudio. No need to exit/enter again.
Change voice / VAD threshold mid-sessionRe-populate AgentParam.role / threshold on the next startAudio; previous segments are unaffected.
Abort teleop mid-flightCall exit(...) directly, skipping any pending doneAudio.
React to SDK auth / disconnectMaintain an authed flag inside AgentAuthCallback.onAuthState; double-check before enter, or drive reconnection when onEnterAck returns code == 1000.
Teleop multiple robots concurrentlyCreate one TeleopRequest per agentId; the SDK indexes by teleopId so they don't collide.