vkb

package
v0.0.0-...-683b059 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 23, 2022 License: BSD-3-Clause Imports: 23 Imported by: 0

README

// Copyright 2021 The Chromium OS Authors. All rights reserved. // Use of this source code is governed by a BSD-style license that can be // found in the LICENSE file.

Handwriting Input Testing

Handwriting.go contains functions used to read SVG files, generate and populate strokes, and draw the SVG file on the handwriting input.

Creating the handwriting input test SVG file

Any SVG editor can be used to create the handwriting input test SVG file, however, this file must be created using LineTo commands only. There are many online editors that specifically use LineTo commands only, or the Line tool can be used to create the file.

Detailed implementation documentation

See go/tast-handwriting-svg-parsing

Documentation

Overview

Package vkb contains shared code to interact with the virtual keyboard.

Index

Constants

This section is empty.

Variables

View Source 
var DragPointFinder = NodeFinder.Role(role.Button).NameContaining("drag to reposition the keyboard")

DragPointFinder returns the finder of the float VK drag button.

View Source 
var KeyFinder = NodeFinder.Role(role.Button)

KeyFinder returns a finder of keys on virtual keyboard.

View Source 
var MultipasteItemFinder = NodeFinder.ClassName("scrim")

MultipasteItemFinder returns a finder of multipaste item on virtual keyboard.

View Source 
var MultipasteSuggestionFinder = NodeFinder.ClassName("chip")

MultipasteSuggestionFinder returns a finder of multipaste suggestion on virtual keyboard header bar.

View Source 
var NodeFinder = nodewith.Ancestor(vkRootFinder).Onscreen().Visible()

NodeFinder returns a finder of node on virtual keyboard. It finds nodes with `offscreen:false` property to avoid finding cached offscreen nodes.

Functions

func KeyByNameIgnoringCase 

func KeyByNameIgnoringCase(keyName string) *nodewith.Finder

KeyByNameIgnoringCase returns a virtual keyboard Key button finder with the name ignoring case.

Types

type HandwritingContext 

type HandwritingContext struct {
	VirtualKeyboardContext
	// contains filtered or unexported fields
}

HandwritingContext represents a context for handwriting.

func (*HandwritingContext) ClearHandwritingCanvas 

func (hwCtx *HandwritingContext) ClearHandwritingCanvas() uiauto.Action

ClearHandwritingCanvas returns an action that clears the handwriting canvas. TODO(b/189277286): Add support to check whether a handwriting canvas is clear for a non-longform canvas.

func (*HandwritingContext) DrawFirstStrokeFromFile 

func (hwCtx *HandwritingContext) DrawFirstStrokeFromFile(filePath string) uiauto.Action

DrawFirstStrokeFromFile returns an action reading the handwriting file, transforming the points into the correct scale, populates the data into the struct, and drawing the first stroke into the handwriting input.

func (*HandwritingContext) DrawStrokesFromFile 

func (hwCtx *HandwritingContext) DrawStrokesFromFile(filePath string) uiauto.Action

DrawStrokesFromFile returns an action reading the handwriting file, transforming the points into the correct scale, populates the data into the struct, and drawing the strokes into the handwriting input.

func (*HandwritingContext) WaitForHandwritingEngineReady 

func (hwCtx *HandwritingContext) WaitForHandwritingEngineReady(checkHandwritingEngineReady uiauto.Action) uiauto.Action

WaitForHandwritingEngineReady returns an action that waits for the handwriting engine to become ready.

type ShiftState 

type ShiftState int

ShiftState describes the shift state of the virtual keyboard. 

const (
	ShiftStateNone ShiftState = iota
	ShiftStateShifted
	ShiftStateLocked
	ShiftStateUnknown
)

Available virtual keyboard shift state. Use ShiftStateUnknown when any errors happen in fetching shift state.

func (ShiftState) String 

func (shiftState ShiftState) String() string

String returns the key representative string content of the shift state.

type VirtualKeyboardContext 

type VirtualKeyboardContext struct {
	// contains filtered or unexported fields
}

VirtualKeyboardContext represents a context of virtual keyboard.

func NewContext 

func NewContext(cr *chrome.Chrome, tconn *chrome.TestConn) *VirtualKeyboardContext

NewContext creates a new context of virtual keyboard.

func (*VirtualKeyboardContext) BackgroundConn 

func (vkbCtx *VirtualKeyboardContext) BackgroundConn(ctx context.Context) (*chrome.Conn, error)

BackgroundConn returns a connection to the virtual keyboard background page, where JavaScript can be executed to simulate interactions with IME.

func (*VirtualKeyboardContext) ClickUntilVKShown 

func (vkbCtx *VirtualKeyboardContext) ClickUntilVKShown(nodeFinder *nodewith.Finder) uiauto.Action

ClickUntilVKShown returns an action retrying left clicks the node until the vk is shown with no error. This is useful for situations where there is no indication of whether the node is ready to receive clicks. The interval between clicks and the timeout can be specified using testing.PollOptions.

func (*VirtualKeyboardContext) DeleteMultipasteItem 

func (vkbCtx *VirtualKeyboardContext) DeleteMultipasteItem(touchCtx *touch.Context, itemName string) uiauto.Action

DeleteMultipasteItem returns an action selecting a multipaste item via longpress and deleting it.

func (*VirtualKeyboardContext) DoubleTapNode 

func (vkbCtx *VirtualKeyboardContext) DoubleTapNode(finder *nodewith.Finder) uiauto.Action

DoubleTapNode returns an action to double tap on a node. Note: DoubleTapNode cannot be replaced by calling TapNode twice. vkbCtx.ui.MouseMoveTo function waits for the node location to be stable. It can take ~500ms and causing long sleep between 2 clicks.

func (*VirtualKeyboardContext) EnableA11yVirtualKeyboard 

func (vkbCtx *VirtualKeyboardContext) EnableA11yVirtualKeyboard(enabled bool) uiauto.Action

EnableA11yVirtualKeyboard returns an action enabling or disabling accessibility mode of the virtual keyboard. When disabled, the tablet non-a11y virtual keyboard will be used.

func (*VirtualKeyboardContext) GetSuggestions 

func (vkbCtx *VirtualKeyboardContext) GetSuggestions(ctx context.Context) ([]string, error)

GetSuggestions returns suggestions that are currently displayed by the virtual keyboard.

func (*VirtualKeyboardContext) GlideTyping 

func (vkbCtx *VirtualKeyboardContext) GlideTyping(keys []string, validateResultFunc uiauto.Action) uiauto.Action

GlideTyping returns a user action to simulate glide typing on virtual keyboard. It works on both tablet VK and A11y VK.

func (*VirtualKeyboardContext) HideVirtualKeyboard 

func (vkbCtx *VirtualKeyboardContext) HideVirtualKeyboard() uiauto.Action

HideVirtualKeyboard returns an action forcing the virtual keyboard to be hidden via Chrome API. It is not recommended to use on testing a real user input through the virtual keyboard. Virtual keyboard should be normally triggered by defocusing an input field. Usage: It can be used in test cleanup.

func (*VirtualKeyboardContext) IsKeyShown 

func (vkbCtx *VirtualKeyboardContext) IsKeyShown(ctx context.Context, keyName string) (bool, error)

IsKeyShown immediately checks whether the given key is shown. TODO (b/182408845) re-implement the function in case an error happens.

func (*VirtualKeyboardContext) IsShown 

func (vkbCtx *VirtualKeyboardContext) IsShown(ctx context.Context) (bool, error)

IsShown immediately checks whether the virtual keyboard is shown. TODO (b/182408845) re-implement the function in case an error happens.

func (*VirtualKeyboardContext) Location 

func (vkbCtx *VirtualKeyboardContext) Location(ctx context.Context) (*coords.Rect, error)

Location returns stable location of the virtual keyboard.

func (*VirtualKeyboardContext) NewHandwritingContext 

func (vkbCtx *VirtualKeyboardContext) NewHandwritingContext(ctx context.Context) (*HandwritingContext, error)

NewHandwritingContext creates a new context for handwriting.

func (*VirtualKeyboardContext) SelectFromSuggestion 

func (vkbCtx *VirtualKeyboardContext) SelectFromSuggestion(candidateText string) uiauto.Action

SelectFromSuggestion returns an action waiting for suggestion candidate (Case Sensitive) to appear and clicks it to select.

func (*VirtualKeyboardContext) SelectFromSuggestionIgnoringCase 

func (vkbCtx *VirtualKeyboardContext) SelectFromSuggestionIgnoringCase(candidateText string) uiauto.Action

SelectFromSuggestionIgnoringCase returns an action waiting for suggestion candidate (Case Insensitive) to appear and clicks it to select.

func (*VirtualKeyboardContext) SetFloatingMode 

func (vkbCtx *VirtualKeyboardContext) SetFloatingMode(uc *useractions.UserContext, enabled bool) uiauto.Action

SetFloatingMode returns an action changing the virtual keyboard to floating/dock layout.

func (*VirtualKeyboardContext) ShiftState 

func (vkbCtx *VirtualKeyboardContext) ShiftState(ctx context.Context) (ShiftState, error)

ShiftState identifies and returns the current VK shift state using left-shift key 'data-key' attribute. Note: It only works on English(US). It works even the VK is not on screen. ShiftLeft: VK is not shifted. ShiftLeft-shift: Vk is Shifted. ShiftLeft-shiftlock: Vk is Shift locked. TODO(b/196272947): Support other input methods other than English(US).

func (*VirtualKeyboardContext) ShowAccessPoints 

func (vkbCtx *VirtualKeyboardContext) ShowAccessPoints() uiauto.Action

ShowAccessPoints returns an action showing the access points panel.

func (*VirtualKeyboardContext) ShowVirtualKeyboard 

func (vkbCtx *VirtualKeyboardContext) ShowVirtualKeyboard() uiauto.Action

ShowVirtualKeyboard returns an action forcing the virtual keyboard show up via Chrome API. It is not recommended to use on testing a real user input through the virtual keyboard. Virtual keyboard should be normally triggered by focusing an input field. Usage: It can be used to test Layout and UI interaction in a quick way. For example, testing switch layout.

func (*VirtualKeyboardContext) SwitchToHandwriting 

func (vkbCtx *VirtualKeyboardContext) SwitchToHandwriting(ctx context.Context) (*HandwritingContext, error)

SwitchToHandwriting changes to handwriting layout and returns a handwriting context.

func (*VirtualKeyboardContext) SwitchToKeyboard 

func (vkbCtx *VirtualKeyboardContext) SwitchToKeyboard() uiauto.Action

SwitchToKeyboard returns an action changing to keyboard layout. TODO(b/195366402): Use test api for switching to keyboard/handwriting mode for VK.

func (*VirtualKeyboardContext) SwitchToMultipaste 

func (vkbCtx *VirtualKeyboardContext) SwitchToMultipaste() uiauto.Action

SwitchToMultipaste returns an action changing to multipaste layout.

func (*VirtualKeyboardContext) SwitchToSymbolNumberLayout 

func (vkbCtx *VirtualKeyboardContext) SwitchToSymbolNumberLayout() uiauto.Action

SwitchToSymbolNumberLayout returns an action changing to symbol number layout.

func (*VirtualKeyboardContext) SwitchToVoiceInput 

func (vkbCtx *VirtualKeyboardContext) SwitchToVoiceInput() uiauto.Action

SwitchToVoiceInput returns an action changing virtual keyboard to voice input layout.

func (*VirtualKeyboardContext) TapAccessPoints 

func (vkbCtx *VirtualKeyboardContext) TapAccessPoints() uiauto.Action

TapAccessPoints returns an action clicking access points button to switch the suggestion bar to layout icons.

func (*VirtualKeyboardContext) TapKey 

func (vkbCtx *VirtualKeyboardContext) TapKey(keyName string) uiauto.Action

TapKey returns an action simulating a mouse click event on the middle of the specified key via a touch event. The key name is case sensitive. It can be any letter of the alphabet, "space" or "backspace".

func (*VirtualKeyboardContext) TapKeyIgnoringCase 

func (vkbCtx *VirtualKeyboardContext) TapKeyIgnoringCase(keyName string) uiauto.Action

TapKeyIgnoringCase returns an action simulating a mouse click event on the middle of the specified key via a touch event. The key name can either be case sensitive or not. It can be any letter of the alphabet, "space" or "backspace".

func (*VirtualKeyboardContext) TapKeyJS 

func (vkbCtx *VirtualKeyboardContext) TapKeyJS(key string) uiauto.Action

TapKeyJS returns an action simulating a tap event on the middle of the specified key via javascript. The key can be any letter of the alphabet, "space" or "backspace".

func (*VirtualKeyboardContext) TapKeyboardLayout 

func (vkbCtx *VirtualKeyboardContext) TapKeyboardLayout() uiauto.Action

TapKeyboardLayout returns an action clicking keyboard layout to switch. The key name is 'Back' in A11y tree.

func (*VirtualKeyboardContext) TapKeys 

func (vkbCtx *VirtualKeyboardContext) TapKeys(keys []string) uiauto.Action

TapKeys return an action simulating tap events in the middle of the specified sequence of keys via touch event. Each key can be any letter of the alphabet, "space" or "backspace". Keys are case sensitive.

func (*VirtualKeyboardContext) TapKeysIgnoringCase 

func (vkbCtx *VirtualKeyboardContext) TapKeysIgnoringCase(keys []string) uiauto.Action

TapKeysIgnoringCase return an action simulating tap events in the middle of the specified sequence of keys via touch event. Each key can be any letter of the alphabet, "space" or "backspace". Keys are case insensitive.

func (*VirtualKeyboardContext) TapKeysJS 

func (vkbCtx *VirtualKeyboardContext) TapKeysJS(keys []string) uiauto.Action

TapKeysJS returns an action simulating tap events on the middle of the specified sequence of keys via javascript. Each keys can be any letter of the alphabet, "space" or "backspace".

func (*VirtualKeyboardContext) TapMultipasteItem 

func (vkbCtx *VirtualKeyboardContext) TapMultipasteItem(itemName string) uiauto.Action

TapMultipasteItem returns an action tapping the item corresponding to itemName in multipaste virtual keyboard.

func (*VirtualKeyboardContext) TapMultipasteSuggestion 

func (vkbCtx *VirtualKeyboardContext) TapMultipasteSuggestion(itemName string) uiauto.Action

TapMultipasteSuggestion returns an action tapping the item corresponding to itemName in multipaste suggestion bar.

func (*VirtualKeyboardContext) TapNode 

func (vkbCtx *VirtualKeyboardContext) TapNode(finder *nodewith.Finder) uiauto.Action

TapNode returns an action to tap on a node. In most cases, TapKey should be primary function for tapping key. This function should only be used when a node can not be unique identified by Name. TODO(b/196273235): Refactor vkb.TapKey function to distinguish keyboard, suggestion bar, node.

func (*VirtualKeyboardContext) UIConn 

func (vkbCtx *VirtualKeyboardContext) UIConn(ctx context.Context) (*chrome.Conn, error)

UIConn returns a connection to the virtual keyboard HTML page, where JavaScript can be executed to simulate interactions with the UI. The connection is lazily created, and this function will block until the extension is loaded or ctx's deadline is reached. The caller should close the returned connection.

func (*VirtualKeyboardContext) WaitForDecoderEnabled 

func (vkbCtx *VirtualKeyboardContext) WaitForDecoderEnabled(enabled bool) uiauto.Action

WaitForDecoderEnabled returns an action waiting for decoder to be enabled or disabled.

func (*VirtualKeyboardContext) WaitForKeysExist 

func (vkbCtx *VirtualKeyboardContext) WaitForKeysExist(keys []string) uiauto.Action

WaitForKeysExist returns an action waiting for a list of keys to appear on virtual keyboard. Note: Should not use FindKeyNode in a loop to implement this function, because it waits for each key within a timeout.

func (*VirtualKeyboardContext) WaitLocationStable 

func (vkbCtx *VirtualKeyboardContext) WaitLocationStable() uiauto.Action

WaitLocationStable returns an action waiting for the virtual keyboard to appear and stable.

func (*VirtualKeyboardContext) WaitUntilHidden 

func (vkbCtx *VirtualKeyboardContext) WaitUntilHidden() uiauto.Action

WaitUntilHidden returns an action waiting for the virtual keyboard to hide. It waits until the node is gone from a11y tree.

func (*VirtualKeyboardContext) WaitUntilShiftStatus 

func (vkbCtx *VirtualKeyboardContext) WaitUntilShiftStatus(expectedShiftState ShiftState) uiauto.Action

WaitUntilShiftStatus waits for up to 5s until the expected VK shift state. Note: It only works on US-en.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.