piaas

README

The Personal IAAS

Piaas is a set of tools that help you develop using multiple machines as if having Personal IAAS.

We all work on laptop today. However, compiling and serving on laptop is crazy. The fan keeps working and creating quite a noise. Your battery drained, cycle count increased. More than ever, peace of mind you lost.

Piaas help your workspace in sync with a remote machine, given it a more powerful desktop, a high end iMac or a shared server. Develop with your favorite IDE, while compiling and serving from the desktop. Your laptop is relaxed to do the thing that absolutely required it to do. Great team work, isn't it ?

Installation

Download the binary from the following link:

• Windows, 64 bit
• Mac OS, 64 bit
• Linux, 64 bit

Place the binary in your path and you're done.

System requirements

The application support Mac OS, but was only tested on Mojave.

Linux should be fine, but untested.

On windows platform, WSL (windows subsystem linux) is required. At the moment Windows 10 is the only version with WSL so is the only version supported.

If you need instructions on how to enable WSL on windows, check this.

Sync

Piaas sync is a wrapper around rsync. It will detect workspace changes and sync necessary files to remote. Piaas is slightly more optimized than running rsync with a file watcher for the following reasons:

1. Piaas detects changed files and update only the changed files.
2. Piaas checks through the ignore file, only necessary files are synced.

Config

Create the following 2 files in your workspace.

piaasconfig.yml

This is the main configuration file, where you describe the connection info of remote machines.

You may configure more than one machines, find samples of different platform in the document/samples folder.

An example of piaasconfig.yml

apiVersion: 1
profiles:
  - name: dev
    connection:
      host: remote
      user: foo
      destination: ~/src

Windows

The piaas configuration on windows platform is trickier than the other. Find the example in the below and note the executable section. This is necessary to execute rsync through WSL. Using other rsync binary may work but was never tested.

Windows example:

apiVersion: 1
executable:
  cmd: wsl
  params: ["rsync"]
profiles:
  - name: dev
    connection:
      host: remote.host.name
      user: foo
      destination: ~/src

.piaasignore

The ignore file uses a very similar format as rsync exclude files. The following notations are supported.

• Exact, path element without special character. Ex: foo
• Anchored, path start with "/". Ex: /foo
• Wildcard, path element that matches any character. Ex: foo*
• Double wildcards, path elements that matches any character including sub directories. Ex: foo/**
• Multi segments, path elements that span over one level in the hierarchy. Ex: foo/bar

An example .piaasignore:

/.idea
/.git
*___jb_tmp___
*___jb_old___
*.class

Prepare

To get piaas sync running, we need to make sure rsync is running. Rsync uses ssh under the hood. The following are required.

• SSH key
• Remote machine was added in known_hosts
• Local public key is authorized in remote machine

To make sure the above work, follow the below steps

1. Make sure you have ssh key

   ls -al ~/.ssh
   # You should have id_rsa and id_rsa.pub files.
   

   If you do not have id_rsa and id_rsa.pub, generate the ssh key

   ssh-keygen
   

2. Connect to remote to make sure the connection works

   # user and remote_host are the connection info from piaasconfig.yml
   ssh user@remote_host
   

   You may be prompted with the below

   The authenticity of host 'remote_host (10.1.1.1)' can't be established.
   ECDSA key fingerprint is SHA256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.
   Are you sure you want to continue connecting (yes/no)?
   

   Say yes to remember the fingerprint.

3. Authorize yourself with remote machine

   ssh-copy-id user@remote_host
   

   The command will ask you to input password. Upon successful authenticated, you'll be authorized to login to remote host without having to input password each time.

   No ssh-copy-id ?

   If your system do not have ssh-copy-id, use your package manager to install it.

   Mac OS: brew install ssh-copy-id
   

Windows

Piaas sync is using WSL (windows subsystem linux). You need to make sure WSL is enabled and execute the above command in WSL bash.

Running

Use the sync command of piaas.

piaas sync <profile name>

The profile name is defined in your piaasconfig.yml file. You may defined multiple profiles, but sync can only work with one profile at a time.

Your workspace will keep sync with the remote server. You may now go to the server to compile and serve from there.

Windows

Piaas sync will invoke rsync through WSL, but it's recommended to run it through normal windows command prompt. If you run it through WSL bash, the command may not be terminated by ctrl-c. You'll therefore have to close the entire command box.

Documentation

Index

• func HandleDebug(c *cli.Context) error
• type App
• type Config
  • func ReadConfig(file string) Config
  • func (conf *Config) GetProfile(name string) (Profile, error)
  • func (conf *Config) GetSyncTarget(name string) (string, error)
• type Event
• type Executable
• type Monitor
  • func NewMonitor(startDir string) Monitor
• type MonitorPath
• type Profile
• type RecursiveMonitor
  • func NewRecursiveMonitor(start string) RecursiveMonitor
  • func (rm *RecursiveMonitor) Start(debounceTime uint64)
  • func (rm *RecursiveMonitor) Stop()
  • func (rm *RecursiveMonitor) Subscribe(subscriber chan<- []string)
  • func (rm *RecursiveMonitor) SubscribeToChanges(subscriber chan<- string)
• type RsyncPattern
  • func NewRsyncPattern(pat string) RsyncPattern
  • func (rp RsyncPattern) Match(path string) bool
  • func (rp RsyncPattern) ToString() string
• type RsyncPatterns
  • func NewRsyncPatterns(basedir string, patterns ...RsyncPattern) RsyncPatterns
  • func (rp *RsyncPatterns) Match(path string) bool
  • func (rp *RsyncPatterns) MatchRelative(rel string) bool
• type RsyncWrapper
  • func NewRsyncWrapper(rsyncCmd Executable, basedir string, target string) RsyncWrapper
  • func (rs *RsyncWrapper) SetIgnoreFile(ignore string)
  • func (rs *RsyncWrapper) SetSshOptions(options string)
  • func (rs *RsyncWrapper) Start(process func(cmd *exec.Cmd))
  • func (rs *RsyncWrapper) SyncAll()
  • func (rs *RsyncWrapper) SyncFiles(files []string)
• type SyncConnection

Functions

func HandleDebug

func HandleDebug(c *cli.Context) error

Turn on debug logging if -debug was specified

Types

type App

type App struct {
	Name    string
	Command string
}

type Config

type Config struct {
	ApiVersion string
	Executable
	Profiles []Profile
	Apps     []App
}

func ReadConfig

func ReadConfig(file string) Config

func (*Config) GetProfile

func (conf *Config) GetProfile(name string) (Profile, error)

func (*Config) GetSyncTarget

func (conf *Config) GetSyncTarget(name string) (string, error)

Find matching profile and use the information to compose the rsync target string. In this format: ${user}@${host}:${target}

type Event

type Event struct {
	Path string
}

type Executable

type Executable struct {
	Cmd    string
	Params []string
}

type Monitor

type Monitor interface {
	// Start watching for file system changes. Specify the debounceTime so the events are collected within the debounce time.
	Start(debounceTime uint64)
	// Register a observer to receive the fs events
	Subscribe(observer chan<- []string)
	// Stop file system watching.
	Stop()
}

func NewMonitor

func NewMonitor(startDir string) Monitor

type MonitorPath

type MonitorPath struct {
	// contains filtered or unexported fields
}

type Profile

type Profile struct {
	// profile name
	Name string
	// The connection information used by sync module
	Connection SyncConnection
	// The name of ignore file, default to .piaasignore
	IgnoreFile string
}

Profile controls how the project can sync to remote machine.

type RecursiveMonitor

type RecursiveMonitor struct {
	// contains filtered or unexported fields
}

func NewRecursiveMonitor

func NewRecursiveMonitor(start string) RecursiveMonitor

Traverse the tree and create monitor for all directory

func (*RecursiveMonitor) Start

func (rm *RecursiveMonitor) Start(debounceTime uint64)

Start watching on all registered monitors, and manage the removal and addition of new directories.

func (*RecursiveMonitor) Stop

func (rm *RecursiveMonitor) Stop()

func (*RecursiveMonitor) Subscribe

func (rm *RecursiveMonitor) Subscribe(subscriber chan<- []string)

func (*RecursiveMonitor) SubscribeToChanges

func (rm *RecursiveMonitor) SubscribeToChanges(subscriber chan<- string)

type RsyncPattern

type RsyncPattern struct {
	// contains filtered or unexported fields
}

Implement basic rsync filter rules.

Support the below notation

- Exact, path element without special character. Ex: foo - Anchored, path start with `/`. Ex: /foo - Wildcard, path element that matches any character. Ex: foo* - Double wildcards, path elements that matches any character including sub directories. Ex: foo/** - Multi segments, path elements that span over one level in the hierarchy. Ex: foo/bar

func NewRsyncPattern

func NewRsyncPattern(pat string) RsyncPattern

Create a new RsyncPattern.

func (RsyncPattern) Match

func (rp RsyncPattern) Match(path string) bool

Match a given path to make sure it matches this pattern or not. You should not directly

func (RsyncPattern) ToString

func (rp RsyncPattern) ToString() string

type RsyncPatterns

type RsyncPatterns struct {
	// contains filtered or unexported fields
}

func NewRsyncPatterns

func NewRsyncPatterns(basedir string, patterns ...RsyncPattern) RsyncPatterns

Create a new RsyncPatterns.

func (*RsyncPatterns) Match

func (rp *RsyncPatterns) Match(path string) bool

Working on a path under the basedir to see if it matches with any of the registered patterns. False is returned if path wasn't under basedir.

func (*RsyncPatterns) MatchRelative

func (rp *RsyncPatterns) MatchRelative(rel string) bool

Working on a path relative to the basedir.

type RsyncWrapper

type RsyncWrapper struct {
	// The rsync command to execute, usually just 'rsync' on Mac and Linux.
	// But may involves batch file on windows.
	Executable
	// contains filtered or unexported fields
}

func NewRsyncWrapper

func NewRsyncWrapper(rsyncCmd Executable, basedir string, target string) RsyncWrapper

Create a RsyncWrapper using `cmd` as the command, running on `basedir`, syncing to `target`. The RsyncWrapper will also be configured to use ssh options 'ConnectTimeout=10'

func (*RsyncWrapper) SetIgnoreFile

func (rs *RsyncWrapper) SetIgnoreFile(ignore string)

func (*RsyncWrapper) SetSshOptions

func (rs *RsyncWrapper) SetSshOptions(options string)

func (*RsyncWrapper) Start

func (rs *RsyncWrapper) Start(process func(cmd *exec.Cmd))

Open the channel to start working on sync events

When cmd is received, `process` will be invoked to actually running the command.

func (*RsyncWrapper) SyncAll

func (rs *RsyncWrapper) SyncAll()

Sync all files to remote.

func (*RsyncWrapper) SyncFiles

func (rs *RsyncWrapper) SyncFiles(files []string)

Sync only the specified files If the files list is empty, do nothing

type SyncConnection

type SyncConnection struct {
	Host        string
	User        string
	Destination string
}

The sync connection target