Help us shape the future of Go! Take the 2026 Go Developer Survey

android

package module
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 25, 2026 License: BSD-3-Clause Imports: 12 Imported by: 0

README

go-xrkit/android

ci Go Reference Go Report Card

Screen capture on Android from pure Go, CGO_ENABLED=0, for an XR compositor that redraws every frame.

if !android.Authorized() {
        ok, err := android.RequestAuthorization(ctx) // the system consent dialog
        ...
}
d, _ := android.DefaultDisplay(ctx)

s, err := android.CaptureDisplay(ctx, d, android.Options{FPS: 60}) // a CEILING
if err != nil {
        return err
}
defer s.Close()

for {
        f, fresh := s.Frame() // BORROWED pixels, no copy, no allocation
        if fresh {
                composite(f.Pix, f.Width, f.Height, f.Stride) // RGBA, stride carried
        }
        render()
}

It is the Android member of a family — go-macos/screencapture, go-freedesktop/screencast, go-mswin/screencapture — deliberately shaped so one consumer drives all of them through near-identical adapters: Displays, CaptureDisplay, Stream.Frame() (Frame, bool) lending borrowed pixels with a stride carried and never assumed, an idempotent Close.

Read this first: what Android will and will not allow

The interesting question for an XR virtual desktop is not "can I capture the screen" — it is "can I create extra desktops and put applications on them", the way go-xrkit/desk does on macOS with real virtual displays.

The answer is no, and it is not close. An ordinary, unprivileged APK may mirror the screen it already has, and may draw its own content on a display that is already attached. It may not manufacture desktops.

Everything below was read off a live Android 15 (API 35, arm64) system, not off documentation. host/../docs carries no prose about this: the transcript is the documentation.

Creating a virtual display and launching something on it
what was asked for what Android 15 answered
VIRTUAL_DISPLAY_FLAG_PUBLIC | PRESENTATION SecurityException: Requires CAPTURE_VIDEO_OUTPUT or CAPTURE_SECURE_VIDEO_OUTPUT permission, or an appropriate MediaProjection token in order to create a screen sharing virtual display.
… | OWN_CONTENT_ONLY created — display id 2, flags=0x8 [PRESENTATION]. Note what is absent: FLAG_TRUSTED.
launch Settings on it, ActivityOptions.setLaunchDisplayId(2) SecurityException: Permission Denial: starting Intent { act=android.settings.SETTINGS … } from ProcessRecord{… org.goxrkit.probe/u0a151} … with launchDisplayId=2
launch our own activity on it the same denial — an app may not put even itself on its own virtual display
… | VIRTUAL_DISPLAY_FLAG_TRUSTED (1 << 10) SecurityException: Requires ADD_TRUSTED_DISPLAY permission to create a trusted virtual display.

VIRTUAL_DISPLAY_FLAG_TRUSTED is not merely undocumented — it is not in the public SDK at all. javap on the API 35 android.jar lists exactly five flags, and TRUSTED is not among them:

$ javap -classpath $ANDROID_HOME/platforms/android-35/android.jar \
        android.hardware.display.DisplayManager | grep VIRTUAL_DISPLAY
  public static final int VIRTUAL_DISPLAY_FLAG_AUTO_MIRROR;
  public static final int VIRTUAL_DISPLAY_FLAG_OWN_CONTENT_ONLY;
  public static final int VIRTUAL_DISPLAY_FLAG_PRESENTATION;
  public static final int VIRTUAL_DISPLAY_FLAG_PUBLIC;
  public static final int VIRTUAL_DISPLAY_FLAG_SECURE;

And the permissions that would unlock it are, on the same device:

$ adb shell pm list permissions -f
  permission:android.permission.ADD_TRUSTED_DISPLAY   protectionLevel:signature|role
  permission:android.permission.CAPTURE_VIDEO_OUTPUT  protectionLevel:signature
  permission:android.permission.CREATE_VIRTUAL_DEVICE protectionLevel:internal|role

signature means "signed with the platform key". internal|role cannot even be granted by signature. None of the three is reachable by an APK you or I can install.

VirtualDeviceManager (Android 14+) does not change the answer

It is the obvious thing to hope for, and it is closed twice over. The public SDK class carries no createVirtualDevice at all — only read-only queries:

$ javap … android.companion.virtual.VirtualDeviceManager
  public java.util.List<VirtualDevice> getVirtualDevices();
  public VirtualDevice getVirtualDevice(int);
  public void registerVirtualDeviceListener(…);
  public void unregisterVirtualDeviceListener(…);

The method does exist on the platform class, so it was called reflectively to get the platform's own answer rather than the header's:

vdm: createVirtualDevice EXISTS on the platform class:
     public VirtualDeviceManager$VirtualDevice createVirtualDevice(int, VirtualDeviceParams)
vdm: createVirtualDevice REFUSED:
     java.lang.SecurityException: Access denied, requires: android.permission.CREATE_VIRTUAL_DEVICE
So what an XR application on Android actually gets

Mirroring, plus its own content. One ribbon carrying the phone's real screen and whatever the application draws itself — not a ring of independent desktops. That is a smaller feature than the macOS one, and pretending otherwise would only produce an elaborate structure around something the platform refuses.

What a capture delivers

Measured, on the emulator named at the bottom of this file, with the cmd/xrcapture binary packaged into the APK:

DISPLAY display 0 "Built-in Screen" 1080x2400 @420dpi 60Hz (default)
capture 1080x2400 stride 4320 slots 3 ceiling 60.0 fps
lent 3 slots of 10368000 bytes
FIRST frame 1080x2400 stride 4320 (width*4 = 4320, padding 0) seq 1
RESULT frames=492 over 11.99s = 41.0 fps; identical-to-previous=68/491
RESULT Frame(): 22.8 ns/op, 0.000 allocations/op over 200000 calls
RESULT stats {Frames:493 Superseded:1 Interval:16.755125ms}
size the display's native pixels, 1080×2400 here — no downscale is imposed
format RGBA_8888 (android.graphics.PixelFormat 1), packed, 4 bytes per pixel
stride 4320 bytes, which happens to be exactly width*4 on this device. The API carries the number anyway, because a graphics allocator is free to pad and a sheared image is the classic result of assuming it does not
rate 41.0 fps sustained at 1080×2400 on a software-rendered emulator, with the capturing app in the background. A second run of the same binary, against a slower-changing screen, measured 18.5 fps — the ceiling is a ceiling, and what the screen does sets the rest. A Java-only probe of the same ImageReader path, with no Go in it, measured 48.0 fps, which bounds this package's own overhead at a few frames a second
Frame() 22.8 ns and 0.000 allocations per call, over 200 000 calls on the device. That is the borrow: Frame.Pix aliases the shared mapping the host wrote into
The content really changes

A capture that hands back a frozen buffer looks exactly like a working one until somebody hashes it, so cmd/xrcapture hashes every frame: 68 of 491 frames were identical to their predecessor and 423 were not, while another application scrolled in front. A run where every frame matched would be reported as a failure, not as a capture.

One frame is saved as a PNG artefact: testdata/artifacts/android-capture.png — 1080×2400, the Settings app scrolling, taken while this app was in the background. The cast indicator in the status bar is Android's own "something is recording your screen" chip.

Frames only arrive when something changes

Like ScreenCaptureKit on macOS, a MediaProjection is change-driven and Options.FPS is a ceiling, not a rate. Left in front of a static screen, the same cmd/xrcapture binary took 35 frames in the first 0.6 seconds and then nothing at all for the remaining eleven — RESULT frames=35 over 0.63s. That is the platform saying nothing moved; the second return value of Frame is the truth about it, not a timer.

Going to the background does not stop it

This matters more here than in most apps, because our app is the thing being covered. The probe deliberately calls moveTaskToBack(true) the instant consent is granted, and then captures another application for twelve seconds: onCapturedContentVisibilityChanged(visible=true), 492 frames, no interruption.

What is required for that is a foreground service of type mediaProjection, and from API 34 it is required anyway: the platform refuses to issue a projection token to a process that is not already running one. FOREGROUND_SERVICE_MEDIA_PROJECTION is protectionLevel:normal, so an ordinary APK simply declares it.

MediaProjectionManager.createScreenCaptureIntent() must be answered by the user, the token is single-use, and nothing remembers the answer — unlike a macOS TCC grant. Authorized() therefore answers "right now", not "ever", and RequestAuthorization exists as a separate call because the dialog takes over the screen and an application should choose when that happens.

Reaching the glasses

On a phone with XR glasses on USB-C DP Alt Mode, Android sees an ordinary second Display. Two routes were exercised against a trusted, public secondary display (see the honesty note below about which one):

  • Presentation — a window of ours on the other display while the phone shows something else. Presentation.show() succeeded, reporting presentation display=8 context size=1920x1080 dpi=320 — the whole display — and dumpsys window confirms it from the outside: Display: mDisplayId=8 … mObscuringWindow=Window{961b015 u0 org.goxrkit.probe}.

  • ActivityOptions.setLaunchDisplayId — this one is the surprise. Launching our own activity worked, and so did launching Settings:

    Display #8 (activities from top to bottom):
    topResumedActivity=ActivityRecord{… com.android.settings/.homepage.SettingsHomepageActivity}
Display #0 (activities from top to bottom):
    topResumedActivity=ActivityRecord{… org.goxrkit.probe/.Probe}

    The framework's rule is "anyone may launch on a public display"; the restriction that bites in the table above applies to virtual displays that are not trusted. So on the one display the glasses provide, an ordinary app can place other applications. It still cannot make a second one.

The display an app-created virtual display gets is flags=0x8 [PRESENTATION]. The system-created secondary display is flags=0x88 [PRESENTATION|TRUSTED], and the built-in panel is 0x4083. The trusted bit is the whole difference.

What a headset's Display reports — its Name is what the catalogue in go-xrkit/xrkit/glasses matches models on. For the built-in panel Android reports a device string ("Built-in Screen" on this emulator); for an external sink it comes from the EDID. No Android artifact for any real headset exists here — see below, and please read that section before believing anything about a specific model.

Input

Nothing new is needed, and that is the finding rather than an omission. A Bluetooth or USB keyboard reaches an Android app as ordinary KeyEvents, which go-widgets/android already forwards to the Go process as MsgKey (Android key code plus the rune its key-character map produced), alongside touch and wheel events. A ribbon scrolled from the keyboard therefore works through the existing back-end with no capture-specific code at all. The emulator enumerates a KEYBOARD | ALPHAKEY | DPAD device, so key injection reaches an app there too.

How it is put together

Android hands no MediaProjection, and no drawable surface, to a process that is not the app: both are behind JNI, and JNI needs cgo. So the app is two processes, exactly as go-widgets/android established:
Java host (host/, ~600 lines over three files) owns the projection token, the ImageReader and the mirroring VirtualDisplay. Forwards bytes. Decides nothing.
Go application an ordinary CGO_ENABLED=0 GOOS=android executable. Owns which display, which size, which rate, and when to stop.
Why a separate repository, and a Service rather than an Activity

An APK may hold many components but only one of them may own the drawing surface, and in a go-widgets application that one is go-widgets/android's GwHostActivity. Screen capture also has no business living inside a widget toolkit. So this is its own module, and its host is:

  • a Service — which is mandatory anyway from API 34, as above;
  • a transparent Activity used for nothing but holding the consent dialog, because consent is an activity result and nothing else.

The two hosts coexist as components of one package, each owning what it must, and the single Go process talks to both over two sockets. Neither host knows the other exists.

That composition has one practical wrinkle, and it is solved here rather than by changing go-widgets: the process is spawned by go-widgets' Activity, which knows nothing about capture and sets no socket name for it. Android gives a spawned process no way to ask its own package name either. What it does give it is HOME — which that host sets to /data/user/0/<package>/files — so android.DeriveSocket reads the package out of it and derives the host's socket name. XR_ANDROID_SOCKET overrides it when something does name one.

  MsgStart ─────────────────────► XrHostService ──► MediaProjection
                                        │             │
  MsgConfig ◄─── stride, slots ─────────┤             ▼
  MsgBuffer ◄─── SCM_RIGHTS fd ─────────┤          ImageReader
                                        │             │
  Frame() ◄── mmap(PROT_READ) ◄── SharedMemory ◄── one memcpy
                                        │
  MsgFrame{seq, slot, w, h, stride} ◄───┘
The frame buffer belongs to the HOST, and that is forced

The natural design — the Go side creates a memfd and lends it to the host, the way go-widgets/android lends its framebuffer — does not work in this direction, because the host has to write. An Android app cannot map a descriptor it received read-write. All three routes were tried on the device:

route Android 15's answer
SharedMemory.fromFileDescriptor(pfd) on a Go memfd IllegalArgumentException: FileDescriptor is not a valid ashmem fd
new RandomAccessFile("/proc/self/fd/" + fd, "rw") FileNotFoundException: /proc/self/fd/112: open failed: EACCES (Permission denied)
FileInputStream/FileOutputStream .getChannel().map(READ_WRITE, …) NonWritableChannelException — each of those channels is open one way only

So the host creates the region itself with SharedMemory.create() — ashmem, which dirties no page cache at all, so tens of megabytes a second of pure scratch never reach flash — and lends the descriptor to the application, which maps it PROT_READ. A stray write on the Go side traps instead of silently corrupting a frame under the compositor.

SharedMemory keeps its descriptor to itself, but writeToParcel is public and writes exactly that descriptor, so a Parcel is the supported way back to one.

Three slots, and why not two

Frame() lends; the host keeps producing. With two slots a host that produced two frames while the consumer held one would overwrite the borrow under it. The default and the minimum are therefore three, which is the same reasoning — and the same number — behind go-macos/screencapture's MinQueueDepth. A slot is a whole framebuffer (10.4 MB for a 1080×2400 phone), so MaxQueueDepth is a real memory limit rather than a formality.

One copy per frame, and there is no zero-copy path

The ImageReader plane is gralloc memory. Mapping that needs the graphics HAL, which is behind cgo — the one thing this stack must not have. So there is exactly one copy: direct ByteBuffer to direct ByteBuffer, which is a native memcpy with no Java heap in the middle and no allocation per frame. Measured on a 1080×2400 frame (10.37 MB): 0.325 ms, about 32 GB/s.

Everything after that is free: the Go side maps the same pages and Frame() returns a subslice.

Building the APK

No Gradle and no Kotlin: the host is three Java files and the application is a Go binary, so the SDK's own tools are the whole tool chain.

export ANDROID_HOME=... JAVA_HOME=...
sdkmanager --install "platforms;android-35" "build-tools;35.0.0"
host/build.sh                                # → host/out/xrhost.apk
adb install host/out/xrhost.apk
adb shell pm grant org.goxrkit.androidhost android.permission.POST_NOTIFICATIONS
adb shell am start -n org.goxrkit.androidhost/org.goxrkit.android.XrDemoActivity
adb logcat -s xrcapture xr-host

APP_BIN=/path/to/binary PACKAGE=org.example.app host/build.sh packages an application from another module; the flags are the same ones go-widgets/android's script takes, because it is the same script.

arm64 is the only CGO-free Android architecture — android/arm, android/amd64 and android/386 all answer "requires external (cgo) linking, but cgo is not enabled" — and CI asserts both halves of that, so the day Go lifts the restriction is a red build rather than a silent one.

Testing

100.0% statement coverage, gated in CI on both lanes — the Linux transport and the off-Android stub. A stub nobody tests is a stub that panics the day somebody links it.

The transport is //go:build linux, and Android is Linux, so the suite runs against a fake host over a real socket, a real memfd and a real SCM_RIGHTS handover — not a mock:

CGO_ENABLED=0 GOOS=android GOARCH=arm64 go test -c -cover -coverpkg=. -o xr.test .
adb push xr.test /data/local/tmp/ && adb shell /data/local/tmp/xr.test

Two tests are worth naming because they assert things a reading of the code cannot:

  • TestFrameIsBorrowedNotCopied writes through the host's mapping and reads it back out of a Frame the consumer already holds. The whole no-allocation promise rests on that frame not being a copy.
  • TestStrayDescriptorsAreNotLeaked throws eighty misplaced file descriptors at the client and then counts the process's own /proc/self/fd. A leaked descriptor is invisible until the process runs out of them, so it is measured rather than reasoned about.

-race runs on the Linux lane only: the race detector needs cgo, which is the very thing an Android application binary must not have.

What was actually tested against

This section distinguishes three things and never rounds upward.

Reproducing all of it
# 1. an arm64 Android 15 emulator (Apple Silicon or an arm64 Linux host)
sdkmanager --install "platforms;android-35" "build-tools;35.0.0" \
           "system-images;android-35;default;arm64-v8a" "emulator"
avdmanager create avd -n xr35 -k "system-images;android-35;default;arm64-v8a"
emulator -avd xr35 -no-window -no-audio -gpu swiftshader_indirect &
adb wait-for-device

# 2. the unit suite, on the device
CGO_ENABLED=0 GOOS=android GOARCH=arm64 go test -c -cover -coverpkg=. -o xr.test .
adb push xr.test /data/local/tmp/
adb shell 'cd /data/local/tmp && chmod +x xr.test && TMPDIR=/data/local/tmp ./xr.test'

# 3. the end-to-end capture, through a real APK
host/build.sh && adb install -r host/out/xrhost.apk
adb shell pm grant org.goxrkit.androidhost android.permission.POST_NOTIFICATIONS
adb shell appops set org.goxrkit.androidhost PROJECT_MEDIA allow   # skips the
                       # consent dialog; an adb-only shortcut, never an app one
adb shell am start -n org.goxrkit.androidhost/org.goxrkit.android.XrDemoActivity
sleep 3 && adb shell am start -a android.settings.SETTINGS    # something moving
for i in $(seq 1 26); do adb shell input swipe 540 1600 540 900 300; \
                         adb shell input swipe 540 900 540 1600 300; done
adb logcat -s xrcapture xr-host
adb pull /storage/emulated/0/Android/data/org.goxrkit.androidhost/files/android-capture.png

# 4. the virtual-display refusals, and the secondary-display results
adb shell settings put global overlay_display_devices "1920x1080/320"
#   … then the probe under host/, whose transcript is quoted above
adb shell pm list permissions -f | grep -A4 ADD_TRUSTED_DISPLAY
javap -classpath "$ANDROID_HOME/platforms/android-35/android.jar" \
      android.hardware.display.DisplayManager | grep VIRTUAL_DISPLAY
Hardware connected and exercised

None. No XR glasses were attached to an Android device for any of this. That is the honest headline and it should be read before anything else here.

Everything above was measured on the Android emulator, API level 35 (Android 15), system-images/android-35/default/arm64-v8a, running on an Apple Silicon Mac — fingerprint Android/sdk_phone64_arm64/emu64a:15/AE3A.240806.019/12368160:userdebug/test-keys, 1080×2400 at 420 dpi. A real arm64 Android system, a real MediaProjection, a real SharedMemory, a real socket, real frames. It is not a real phone, and it is certainly not a phone with glasses on its USB-C port.

Partially observed
  • A secondary display. The Presentation and setLaunchDisplayId results come from the emulator's simulated secondary display (settings put global overlay_display_devices "1920x1080/320"), which reports type=OVERLAY, uniqueId="overlay:1", FLAG_PRESENTATION|FLAG_TRUSTED. A display attached over DP Alt Mode is type=EXTERNAL and is likewise public and trusted, and the framework's launch check keys on virtual versus not and on private/trusted rather than on the type — so the result should carry. It was not verified on a real external display, and "should carry" is not "verified".
  • Frame rates. 41–48 fps at 1080×2400 was measured on a software-rendered emulator. A device with a GPU-composited surface should do better, but this repository has not measured one.
Known only from documentation
  • Every published claim about a specific headset — display name, USB identity, the modes it offers — including everything in go-xrkit/xrkit/glasses. There is no Android artifact for any real headset in this repository at all. A macOS display name is not an Android one, and neither is an EDID read on Linux.
  • That VITURE's own SpaceWalker uses this route on a phone over DP Alt Mode.
Send us hardware

We will gladly add and verify a model we can hold. If you want a device supported and verified rather than quoted, send us the hardware and it will be tested against and listed here. Failing that, the next best thing is to plug one in yourself and tell us what adb shell dumpsys display said.

Known gaps

Deliberate, and stated rather than hidden:

  • no real device, and no real glasses — see above;
  • capture of a second display is impossible for an unprivileged app and is reported as ErrNotCapturable rather than attempted. MediaProjection mirrors the default display; anything else needs CAPTURE_VIDEO_OUTPUT, which is signature;
  • Presentation is not wired into this package. Drawing on the glasses is the widget back-end's job, not the capture package's; what this repository contributes is the measured fact that the route exists and what it costs;
  • the row stride is measured, not guessed — but there is a fallback. The host waits up to two seconds for its first Image to learn the allocator's real stride. If none arrives (a screen that is off, say) it announces an aligned upper bound instead, and refuses — loudly, with MsgStopped — any later frame that would not fit. That path has not been provoked on a device.

License

BSD-3-Clause. See LICENSE.

Documentation

Overview

Package android captures the Android screen from a pure-Go, CGO-free application, for an XR compositor that redraws every frame.

What Android permits, and what it does not

Everything here is shaped by one measured fact: an ordinary, unprivileged APK may MIRROR the screen and may draw its own content on an attached display, and it may do nothing else. It cannot give itself extra desktops. On Android 15 (API 35), creating a virtual display and launching an activity on it fails with 

SecurityException: Permission Denial: starting Intent { ... }
    ... with launchDisplayId=N

for ANY activity, including the caller's own; asking for VIRTUAL_DISPLAY_FLAG_TRUSTED — which is not in the public SDK at all — fails with "Requires ADD_TRUSTED_DISPLAY permission", a signature|role permission; and VirtualDeviceManager.createVirtualDevice fails with "Access denied, requires: android.permission.CREATE_VIRTUAL_DEVICE", which is internal|role. See the README for the whole transcript. So on Android an XR ribbon carries the phone's real screen, and the application's own content, and that is all.

The split, and why there is a Java host

Android hands no drawable surface, and no MediaProjection, to a process that is not the app: both are behind JNI, and JNI needs cgo. So the app is two processes, exactly as in go-widgets/android — a small Java host owning the Android objects, and an ordinary CGO_ENABLED=0 GOOS=android executable owning everything above the pixels. This package is the application half.

The hot path

Stream.Frame hands back a BORROWED view of the most recent frame: the bytes are the shared mapping the host wrote into, not a copy, and the borrow lasts until the next Stream.Frame, Stream.WaitFrame or Stream.Close. In steady state a Frame call performs no allocation at all. Copy out with Frame.CopyTight or Frame.NRGBA to keep it longer.

Stride

A captured frame's rows may be PADDED: Stride is the number of BYTES per row and it is not necessarily Width*4. Always index with Stride, or use Frame.Row. The Android 15 emulator measured here reported stride 4320 for a 1080-wide capture — exactly Width*4, no padding — which is precisely why the API carries the number instead of trusting it.

Frames only arrive when something changes

A MediaProjection capture is change-driven, like ScreenCaptureKit on macOS. Options.FPS is a CEILING, not a rate: with a still screen, this package's own probe took 39 frames in 4.0 s and then nothing at all; with the screen scrolling it took 574 frames in 11.9 s. Do not treat a missing frame as a failure; the second return value of Stream.Frame is the truth about whether anything changed.

Every capture session needs the user to agree through a system dialog, and the agreement dies with the session. Authorized reports whether the host already holds a projection token; RequestAuthorization asks the host to put the dialog in front of the user and blocks until they answer.

Index

Constants

View Source 
const (
	// MsgDisplays answers MsgListDisplays with everything the host can see.
	MsgDisplays uint8 = 0x01
	// MsgConfig announces a started capture: the frame geometry, the pixel
	// format and the layout of the shared buffer the frames will appear in.
	MsgConfig uint8 = 0x02
	// MsgFrame says a frame has been written into one slot of the shared
	// buffer. It carries no pixels: the pixels are already there.
	MsgFrame uint8 = 0x03
	// MsgStopped says the capture ended, and why. The user revoking the
	// projection from the system UI arrives here, not as an error on a call.
	MsgStopped uint8 = 0x04
	// MsgConsent answers MsgConsentRequest with the user's decision.
	MsgConsent uint8 = 0x05
	// MsgError reports a failure the host could not answer in-band.
	MsgError uint8 = 0x06
	// MsgBuffer hands the application the shared frame buffer, as an ancillary
	// descriptor, once a capture has been configured. It is the ONE message
	// carrying a descriptor, so it is always written in a single sendmsg.
	//
	// The buffer belongs to the HOST, not to the application, and that is
	// forced rather than chosen: an Android app cannot map a descriptor it
	// received read-write. SharedMemory.fromFileDescriptor rejects a Go memfd
	// outright ("FileDescriptor is not a valid ashmem fd"), and reopening it
	// through /proc/self/fd is refused with EACCES. Only a region the host
	// itself created with SharedMemory.create is writable by the host, so the
	// host creates it and lends it here.
	MsgBuffer uint8 = 0x07

	// MsgListDisplays asks the host what displays exist.
	MsgListDisplays uint8 = 0x81
	// MsgConsentRequest asks the host to put the screen-capture consent dialog
	// in front of the user. Only the host can: consent is an Activity result,
	// and the application owns no Activity.
	MsgConsentRequest uint8 = 0x82
	// MsgStart asks the host to begin capturing.
	MsgStart uint8 = 0x83
	// MsgStop asks the host to end the capture but stay connected.
	MsgStop uint8 = 0x84
	// MsgBye tells the host the application is going away.
	MsgBye uint8 = 0x85
)

Message types. Host→app messages are below 0x80 and app→host at or above it, so a misrouted message is a decode error rather than a plausible other message.

View Source 
const (
	// StopUser is the user revoking the projection — from the cast chip in the
	// status bar, or from the notification the foreground service must show.
	// It is the normal way a capture ends and is NOT a malfunction.
	StopUser uint8 = 0
	// StopSystem is the platform stopping us: a new projection started
	// elsewhere, the device locked, the service was killed.
	StopSystem uint8 = 1
	// StopApp is this application having asked, with MsgStop or MsgBye.
	StopApp uint8 = 2
)

Reasons a capture stopped, carried by StoppedMessage.

View Source 
const (
	FlagSupportsProtectedBuffers uint32 = 1 << 0
	FlagSecure                   uint32 = 1 << 1
	FlagPrivate                  uint32 = 1 << 2
	FlagPresentation             uint32 = 1 << 3
	FlagRound                    uint32 = 1 << 4
)

Display flags, mirroring android.view.Display's public FLAG_* constants. The two that matter to an XR application are FlagPresentation, which says the system considers a display suitable for an app's own full-screen content, and FlagPrivate, which says it belongs to one app.

View Source 
const (
	// DefaultFPS is the frame-rate ceiling used when Options.FPS is zero.
	DefaultFPS = 60.0
	// DefaultQueueDepth is the slot count used when Options.QueueDepth is zero.
	DefaultQueueDepth = 3
	// MinQueueDepth is the smallest slot count this package accepts.
	MinQueueDepth = 3
	// MaxQueueDepth bounds the slot count. A slot is a whole framebuffer —
	// 10.4 MB for a 1080x2400 phone — so this is a real memory limit, not a
	// formality.
	MaxQueueDepth = 8
	// MaxDimension is the largest frame edge accepted, a sanity bound well
	// above any real display.
	MaxDimension = 32768
)

Defaults applied to the zero value of the corresponding Options field.

View Source 
const DefaultDisplayID = 0

DefaultDisplayID is android.view.Display.DEFAULT_DISPLAY.

View Source 
const EnvSocket = "XR_ANDROID_SOCKET"

EnvSocket names the environment variable carrying the abstract socket the XR host listens on. A host that spawns this process sets it.

It is not the only route, and in the composition this package is really for it is not the route taken: the process is spawned by go-widgets/android's Activity, which owns the drawing surface, knows nothing about capture and sets no such variable. DeriveSocket covers that case from HOME.

View Source 
const MaxPayload = 1 << 16

MaxPayload bounds one decoded message body. The largest message a host legitimately sends is a display list, so a frame beyond this is a desynchronised stream — refused rather than allocated.

View Source 
const SocketSuffix = ".xr"

SocketSuffix is appended to the application's package name to form the abstract socket the host listens on when nothing names one explicitly. It matches XrHostService.SOCKET_SUFFIX.

Variables

View Source 
var (
	// ErrUnsupported is reported on every platform that is not Android, and on
	// Android when the process was not started by a host that speaks this
	// protocol. The module still builds and vets everywhere, so an application
	// keeps cross-building on a developer's workstation.
	ErrUnsupported = errors.New("android: screen capture is unavailable " +
		"(not running under a go-xrkit Android host)")

	// ErrPermissionDenied is reported when the user refused the screen-capture
	// consent dialog, or revoked a projection already running. Its message
	// names the remedy.
	ErrPermissionDenied = errors.New("android: screen-capture consent denied — " +
		"Android asks the user for permission once per capture session and the " +
		"answer cannot be remembered; call RequestAuthorization again and accept " +
		"the system dialog")

	// ErrNoDisplay is reported when a capture was asked for and the host
	// listed no display at all.
	ErrNoDisplay = errors.New("android: no capturable display")

	// ErrNotFound is reported when a display ID does not name anything the
	// host can see.
	ErrNotFound = errors.New("android: no such display")

	// ErrNotCapturable is reported for a display that exists but that an
	// unprivileged app may not capture. MediaProjection mirrors the DEFAULT
	// display and nothing else: capturing a second display needs
	// CAPTURE_VIDEO_OUTPUT, which is a signature permission.
	ErrNotCapturable = errors.New("android: only the default display can be captured " +
		"by an unprivileged app (capturing another display needs the signature " +
		"permission CAPTURE_VIDEO_OUTPUT)")

	// ErrClosed is reported by every [Stream] method after [Stream.Close].
	ErrClosed = errors.New("android: stream is closed")

	// ErrNoFrame is reported by [Stream.WaitFrame] when no frame arrived
	// before its context expired. It is NOT a malfunction: a motionless screen
	// legitimately produces no frames.
	ErrNoFrame = errors.New("android: no frame available")

	// ErrInvalidOption is reported by [Options.Validate] and wraps a
	// description of the offending field.
	ErrInvalidOption = errors.New("android: invalid option")

	// ErrShortBuffer is reported by [Frame.CopyTight] when the destination is
	// too small to hold the frame.
	ErrShortBuffer = errors.New("android: destination buffer too short")
)

Sentinel errors. All are stable and may be matched with errors.Is.

View Source 
var ErrBadPayload = errors.New("android: malformed message payload")

ErrBadPayload reports a message whose body is structurally impossible — a string longer than the body that holds it, a negative count.

View Source 
var ErrShortPayload = errors.New("android: truncated message payload")

ErrShortPayload reports a message whose body is too short for its type.

Functions

func Authorized 

func Authorized() bool

Authorized reports whether the host already holds a projection token, so a capture can start without a dialog. It never prompts.

Unlike a macOS TCC grant this is NOT a lasting permission: Android issues a token per session and forgets it, so this answers "right now", not "ever".

func Available 

func Available() bool

Available reports whether this process is running under a host that speaks this protocol. It consults the environment only: it opens nothing and blocks on nothing, so it is safe to call before deciding whether Android capture is this program's platform at all.

func DecodeBuffer 

func DecodeBuffer(b []byte) (slots int, slotSize int64, err error)

DecodeBuffer decodes the layout of the shared buffer.

func DecodeConsent 

func DecodeConsent(b []byte) (bool, error)

DecodeConsent decodes the user's decision.

func DeriveSocket 

func DeriveSocket(env string, envOK bool, home string) string

DeriveSocket works out which abstract socket to dial.

An explicit name wins: a host that spawned this process sets EnvSocket and that is the end of it. But in the composition this package is really for, the process is spawned by a DIFFERENT host — go-widgets/android's Activity, which owns the drawing surface and knows nothing about capture — and never sets it. Android gives such a process no way to ask its own package name either.

What it does give it is HOME, which that host sets to the app's private files directory: "/data/user/0/org.example.app/files". The package name is in there, and the host's socket name follows from it. That is why the two hosts can live in one APK without either one having to know about the other.

It returns "" when neither route yields a name.

func EncodeBuffer 

func EncodeBuffer(slots int, slotSize int64) []byte

EncodeBuffer encodes the layout of the shared buffer accompanying MsgBuffer.

func EncodeConfig 

func EncodeConfig(c ConfigMessage) []byte

EncodeConfig encodes a ConfigMessage.

func EncodeConsent 

func EncodeConsent(granted bool) []byte

EncodeConsent encodes the user's decision.

func EncodeDisplays 

func EncodeDisplays(ds []Display) []byte

EncodeDisplays encodes a display list.

func EncodeError 

func EncodeError(e ErrorMessage) []byte

EncodeError encodes an ErrorMessage.

func EncodeFrame 

func EncodeFrame(f FrameMsg) []byte

EncodeFrame encodes a FrameMsg.

func EncodeStart 

func EncodeStart(s StartMessage) []byte

EncodeStart encodes a StartMessage.

func EncodeStopped 

func EncodeStopped(s StoppedMessage) []byte

EncodeStopped encodes a StoppedMessage.

func FrameMessage 

func FrameMessage(typ uint8, body []byte) []byte

FrameMessage returns one framed message: a 4-byte big-endian length covering the type byte and the body, then the type byte, then the body.

It exists as bytes rather than as writes because a message that carries an ancillary descriptor has to reach the host in ONE sendmsg: split across two writes, the host could attribute the descriptor to the wrong message.

func ReadMessage 

func ReadMessage(r io.Reader) (typ uint8, body []byte, err error)

ReadMessage reads one framed message. It returns io.EOF when the stream ends cleanly between messages, so a caller can tell a closed host from a truncated one.

func RequestAuthorization 

func RequestAuthorization(ctx context.Context) (bool, error)

RequestAuthorization asks the host to put Android's screen-capture consent dialog in front of the user, and blocks until they answer or ctx expires.

It is a separate call rather than something CaptureDisplay does implicitly because the dialog takes over the screen: an application wants to choose when that happens, and to say why first.

func WriteMessage 

func WriteMessage(w io.Writer, typ uint8, body []byte) error

WriteMessage writes one framed message.

Types

type ConfigMessage 

type ConfigMessage struct {
	// Width and Height are the frame size in pixels.
	Width, Height int
	// Stride is the number of BYTES per row in the shared buffer. It is the
	// host's ImageReader row stride and is NOT necessarily Width*4.
	Stride int
	// Format is the pixel layout, always [FormatRGBA] today.
	Format PixelFormat
	// Slots is how many frame slots the shared buffer holds.
	Slots int
	// SlotSize is the byte size of one slot, Stride*Height rounded up to the
	// host's alignment.
	SlotSize int64
	// DisplayID names the display being captured.
	DisplayID int
}

ConfigMessage announces a started capture.

func DecodeConfig 

func DecodeConfig(b []byte) (ConfigMessage, error)

DecodeConfig decodes a ConfigMessage.

type Display 

type Display struct {
	// ID is the android.view.Display display id. 0 is the built-in screen.
	ID int
	// Name is what the system calls the display. For the built-in panel this
	// is a device string ("Built-in Screen" on the Android 15 emulator); for a
	// display attached over USB-C DP Alt Mode it comes from the sink's EDID,
	// which is how XR glasses announce their model.
	Name string
	// Width and Height are the display's real size in pixels.
	Width, Height int
	// DensityDPI is the display's density in dots per inch.
	DensityDPI int
	// RefreshRate is the active mode's rate in hertz.
	RefreshRate float64
	// Flags are android.view.Display's FLAG_* bits; see [FlagPresentation].
	Flags uint32
}

Display is a display the host can see.

Width and Height are in PIXELS — Android reports no separate point space, so unlike the macOS sibling there is only one pair of numbers, and DensityDPI carries what a scale factor would carry elsewhere.

func DecodeDisplays 

func DecodeDisplays(b []byte) ([]Display, error)

DecodeDisplays decodes a display list.

func DefaultDisplay 

func DefaultDisplay(ctx context.Context) (Display, error)

DefaultDisplay returns the built-in screen, the only display an unprivileged app may capture.

func Displays 

func Displays(ctx context.Context) ([]Display, error)

Displays returns every display the host can see.

Only the default one can be captured (see ErrNotCapturable), but the others matter all the same: a headset attached over USB-C DP Alt Mode appears here, and its Display.Name is how a catalogue identifies the model.

func (Display) Default 

func (d Display) Default() bool

Default reports whether this is the built-in screen, the only display an unprivileged app may capture.

func (Display) Presentation 

func (d Display) Presentation() bool

Presentation reports whether the system considers this display suitable for an application's own full-screen content — which is what an XR headset attached over DP Alt Mode is for.

func (Display) String 

func (d Display) String() string

String renders the display for logs.

type ErrorMessage 

type ErrorMessage struct {
	// Code is the host's own numeric code, 0 when it has none.
	Code int
	// Op names what failed, e.g. "getMediaProjection".
	Op string
	// Detail is the platform's message, e.g. a SecurityException's text.
	Detail string
}

ErrorMessage is a failure the host could not answer in-band.

func DecodeError 

func DecodeError(b []byte) (ErrorMessage, error)

DecodeError decodes an ErrorMessage.

func (ErrorMessage) Error 

func (e ErrorMessage) Error() string

Error renders the operation, the code and the platform's message.

type Frame 

type Frame struct {
	// Pix is the frame's bytes in [FormatRGBA], Stride bytes per row, Height
	// rows. len(Pix) == Stride*Height.
	Pix []byte
	// Width and Height are the frame's size in pixels.
	Width, Height int
	// Stride is the number of BYTES per row. It may be padded and is NOT
	// necessarily Width*4.
	Stride int
	// Seq counts frames since the stream started; it is 0 before the first
	// frame and strictly increases afterwards.
	Seq uint64
	// At is when the host's ImageReader delivered the frame.
	At time.Time
}

Frame is a BORROWED view of one captured frame.

Pix aliases the shared mapping the host wrote into. It stays valid only until the next Stream.Frame, Stream.WaitFrame or Stream.Close on the stream that produced it. Do not retain it; copy with Frame.CopyTight or Frame.NRGBA if you need it to outlive the borrow.

func (Frame) CopyTight 

func (f Frame) CopyTight(dst []byte) (int, error)

CopyTight copies the frame into dst with the row padding removed, so dst holds Width*4*Height bytes of contiguous RGBA. It reports how many bytes it wrote, or ErrShortBuffer if dst is too small. It allocates nothing.

func (Frame) NRGBA 

func (f Frame) NRGBA() (*image.NRGBA, error)

NRGBA copies the frame into a freshly allocated image.NRGBA. It is the convenience path for saving a frame to disk; it allocates, so it does not belong in a per-frame loop.

Android's RGBA_8888 is already image.NRGBA's byte order, so unlike the macOS sibling this is a copy with no channel swap — but it is still a copy, because the source is padded and borrowed.

func (Frame) Row 

func (f Frame) Row(y int) []byte

Row returns row y of the frame, Width*4 bytes with the padding trimmed off. It does not allocate. It returns nil for an out-of-range y or an invalid frame.

func (Frame) TightLen 

func (f Frame) TightLen() int

TightLen is the number of bytes the frame occupies with no row padding, Width*4*Height.

func (Frame) Valid 

func (f Frame) Valid() bool

Valid reports whether the frame holds pixels.

type FrameMsg 

type FrameMsg struct {
	// Seq counts frames since the capture started, from 1.
	Seq uint64
	// Slot is which slot of the shared buffer holds it.
	Slot int
	// Width, Height and Stride describe THIS frame. They normally repeat the
	// config, but the host may resize a capture in flight — a phone rotating
	// under a capture does exactly that — so every frame carries its own
	// geometry rather than inheriting one that may be stale.
	Width, Height, Stride int
	// AtUnixNano is when the host's ImageReader delivered it.
	AtUnixNano int64
}

FrameMsg says a frame has been written into a slot of the shared buffer.

func DecodeFrame 

func DecodeFrame(b []byte) (FrameMsg, error)

DecodeFrame decodes a FrameMsg.

func (FrameMsg) At 

func (f FrameMsg) At() time.Time

At renders the delivery instant. A zero AtUnixNano yields the zero Time rather than the epoch, so "the host did not say" stays distinguishable.

type Options 

type Options struct {
	// Width and Height are the requested frame size in PIXELS. Zero means the
	// display's native size.
	Width, Height int

	// FPS is the CEILING on the frame rate, not a guarantee: a MediaProjection
	// only produces a frame when the content changed. Zero means [DefaultFPS].
	FPS float64

	// QueueDepth is how many frame slots the shared buffer holds. Zero means
	// [DefaultQueueDepth]. It must leave room for the frame lent out and the
	// one being written, so values below [MinQueueDepth] are rejected — with
	// two slots a host that produced two frames while the consumer held one
	// would overwrite the borrow.
	QueueDepth int
}

Options configures a capture stream.

The zero Options is usable: it captures the default display at its native pixel size, at up to 60 frames per second.

func (Options) Validate 

func (o Options) Validate() error

Validate reports whether the options are self-consistent, wrapping ErrInvalidOption. It does not consult the system.

type PixelFormat 

type PixelFormat uint32

PixelFormat names the layout of a captured frame, using Android's own android.graphics.PixelFormat constants so the value on the wire is the one the host read off the ImageReader. 

const FormatRGBA PixelFormat = 1 // android.graphics.PixelFormat.RGBA_8888

FormatRGBA is 32-bit RGBA, android.graphics.PixelFormat.RGBA_8888. It is the only format this package streams, and it is what ImageReader produces for a MediaProjection mirror. It is also byte-for-byte what a go-widgets painter writes, so a captured frame composites with no conversion.

This differs from the sibling packages on other platforms: macOS ScreenCaptureKit hands back BGRA. Frame.NRGBA hides the difference, and anything compositing raw bytes must ask Stream.Format.

func (PixelFormat) BytesPerPixel 

func (f PixelFormat) BytesPerPixel() int

BytesPerPixel is the size of one pixel in this format, 0 for a format this package does not stream.

func (PixelFormat) String 

func (f PixelFormat) String() string

String renders the format for logs.

type StartMessage 

type StartMessage struct {
	// DisplayID is the display to capture. Only the default display can be
	// captured by an unprivileged app; see [ErrNotCapturable].
	DisplayID int
	// Width and Height are the requested frame size in pixels; zero means the
	// display's native size.
	Width, Height int
	// MilliFPS is the frame-rate ceiling in thousandths of a frame per second,
	// so 60 fps is 60000 and a rate below one frame a second is still exact.
	MilliFPS uint32
	// Slots is how many frame slots the shared buffer holds.
	Slots int
}

StartMessage asks the host to begin capturing.

func DecodeStart 

func DecodeStart(b []byte) (StartMessage, error)

DecodeStart decodes a StartMessage.

type Stats 

type Stats struct {
	// Frames is the number of frames the host delivered.
	Frames uint64
	// Superseded is the number of delivered frames replaced by a newer one
	// before the consumer ever asked for them. A large value next to Frames
	// means the consumer is slower than the capture.
	Superseded uint64
	// Last is when the most recent frame arrived.
	Last time.Time
	// Interval is the gap between the two most recent frames.
	Interval time.Duration
}

Stats reports what a stream has seen since it started.

func (Stats) FPS 

func (s Stats) FPS() float64

FPS is the instantaneous rate implied by Stats.Interval, 0 when fewer than two frames have arrived.

type StoppedMessage 

type StoppedMessage struct {
	// Reason is one of [StopUser], [StopSystem] or [StopApp].
	Reason uint8
	// Detail is the host's own wording, which may be empty.
	Detail string
}

StoppedMessage says a capture ended and why.

func DecodeStopped 

func DecodeStopped(b []byte) (StoppedMessage, error)

DecodeStopped decodes a StoppedMessage.

func (StoppedMessage) String 

func (s StoppedMessage) String() string

String renders the reason and the detail for a log or an error.

type Stream 

type Stream struct {
	// contains filtered or unexported fields
}

Stream is a running capture.

func CaptureDisplay 

func CaptureDisplay(ctx context.Context, d Display, o Options) (*Stream, error)

CaptureDisplay starts streaming a display.

The user must already have consented — see RequestAuthorization — and only the default display can be captured. The returned stream must be closed.

func (*Stream) Close 

func (st *Stream) Close() error

Close ends the capture and releases the shared mapping. It is idempotent, and every borrowed frame is invalid afterwards.

func (*Stream) Err 

func (st *Stream) Err() error

Err reports why the SYSTEM stopped the capture, nil while it is running. The user revoking the projection arrives here wrapping ErrPermissionDenied.

func (*Stream) Format 

func (st *Stream) Format() PixelFormat

Format returns the pixel layout of every frame this stream produces.

func (*Stream) Frame 

func (st *Stream) Frame() (Frame, bool)

Frame returns the most recent frame and whether it is newer than the one the previous call returned.

The bytes are BORROWED from the shared mapping and stay valid until the next Frame, WaitFrame or Close. In steady state this allocates nothing.

func (*Stream) Options 

func (st *Stream) Options() Options

Options returns the options actually in force, with every zero field resolved.

func (*Stream) Size 

func (st *Stream) Size() (int, int)

Size returns the capture's frame size in pixels.

func (*Stream) Stats 

func (st *Stream) Stats() Stats

Stats reports what the stream has seen since it started.

func (*Stream) String 

func (st *Stream) String() string

String renders the stream for logs.

func (*Stream) WaitFrame 

func (st *Stream) WaitFrame(ctx context.Context) (Frame, error)

WaitFrame blocks until a frame newer than the last one returned arrives, or ctx expires. It reports ErrNoFrame on expiry, which on a motionless screen is the ordinary answer rather than a malfunction.

Source Files

View all Source files

Directories

Path Synopsis
cmd
xrcapture command
Command xrcapture is the on-device proof for github.com/go-xrkit/android: an ordinary CGO_ENABLED=0 GOOS=android binary that asks the host for consent, captures the screen, and reports what it actually got.
 Command xrcapture is the on-device proof for github.com/go-xrkit/android: an ordinary CGO_ENABLED=0 GOOS=android binary that asks the host for consent, captures the screen, and reports what it actually got.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.