README
¶
DPhoto
Backup photo to your private AWS Cloud, and share them with family and friends. Core features:
| Feature | Version | Description |
|---|---|---|
| Backup medias | v0.1 | Backup photos and videos from USB Sticks, Flash drives, and Camera (USB) when they're plugged |
| Organise by album | v0.1 | Photos and videos are organised by album based on their creation date. Each album is a directory in S3. |
| Migration script | v0.1 | Medias already uploaded in S3 are re-ordered by albums and de-duplicated (with interactive command line interface) |
| Web viewer | v0.5 | See photos by albums, tag them, search by tag or dates |
| Faster UI | v1.0 | Pre-generate miniatures, supports browser caching (miniature + medium quality) and backend cache medium-quality images. |
| Album sharing | v1.3 | Albums can be shared to other users |
| Upgraded sharing | - | Pre-selected emails and globals views, merge owned and shared-with-me albums |
| Upgraded album views | - | Albums are listed by most recent medias, medias can be listed by dates (without being grouped by albums |
| Upgraded backup | - | Backup directly to the cloud (WebDAV), cleaning/filtering possible after backup |
| Photo Frame | - | Generate a random pick of media for a photo frame |
| Album contribution | - | Several users can contribute to the same album (families, friend group) |
| API Driven | - | Provide enhanced API to tip the business on the backend side: CLI authenticated by Google Token (vs AWS Credentials) |
| Android App | - | Minimalist app to synchronise local medias to DPhoto. |
| Tagging | - | Adding tags to medias to find them later, share them, or print them |
| Google Photo | - | Support Google Photo to push images and video, or import from Google Photo. |
Getting Started
Install 'dphoto' command line interface and configure it using the following:
go install github.com/thomasduchatelle/dphoto/cmd/...@latest
dphoto configure
Then use command backup to upload media in a directory, or scan to interactively organise the albums.
dphoto scan /x/y/z
dphoto backup /x/y/z
Contribute
Components:
deployments/infra-data: terraform project to create required infrastructure on AWS for the CLI to work. Project won't be re-usable in a different context without overriding backend and some other defaults.pkg: core domain model and business logic from Hexagonal Architecture. This domain is used from both CLI and app's APIs- DPhoto CLI: installed on the end-user computer, backup photos and videos using command line interface
- APP: deployed on top of
infra-data, contains the APP (API deployed on AWS lambdas, and WEB)
Note: the repository follow https://github.com/golang-standards/project-layout structure convention.
Install development environment
Required tools:
- Infra:
- terraform:
brew install tfenv, Makefile - Serverless Framework:
npm install -g serverless - AWS CLI:
brew install awscli
- terraform:
- Languages & build tools:
make- GoLang:
brew install golang - Yarn:
brew install yarn
- Docker and Docker Compose
brew install docker docker-compose
Setup the environment:
make setup all
# Run tests & build (all sub-projects)
make
Commit messages & Github Actions
Commit messages are following the convention:
<domain>[/<area>] [+tags] - <message>
With:
- domains: catalog, archive, backup
- area: web, cli, proj, api, ...
- supported tags are:
+update-snapshots: snapshots used to test visual regression will be updated+patch,+minor,+major: control the version of the software+pr: creates a Pull Request if the pipeline is successful
Releasing process
Bootstrap an environment with built-in command (one-of pre-requisite):
go run ./tools/infra-bootstrap -domain <domain> -email <email> -env dev -google-client-id <id>
To release a new version:
-
make changes on a feature branch and bump the CLI version:
./scripts/pre-release.sh 1.5.0 -
create a pull request to
next, review the terraform plan and tests then merge -> it will deploy to https://dphoto-dev.duchatelle.net -
create a pull request
next -> main, review the terraform plan then merge -> it will deploy to https://dphoto.duchatelle.net and create a tag for the CLI -
(optional) update local versions of
dphotoby runninggo install github.com/thomasduchatelle/dphoto/cmd/...@latest -
to avoid confusion, next development iteration can be started by running
./ci/pre-release.sh 1.6.0-alpha.
AWS Support
DPhoto only supports AWS to be deployed as a serverless application using AWS Gateway, Lambdas, DynamoDB, and S3.
DynamoDB is a single table documented in README.md.
Required Upgrades
- go cli is using AWS SDK 1.x and should use 2.x
- React -> NextJS
Directories
¶
| Path | Synopsis |
|---|---|
|
api
|
|
|
lambdas
Module
|
|
|
app
|
|
|
viewer_api
Module
|
|
|
cmd
|
|
|
dphoto/config
Package config create AWS session and distribute it (alongside other config value) to any package interested.
|
Package config create AWS session and distribute it (alongside other config value) to any package interested. |
|
delegate
module
|
|
|
dphoto
module
|
|
|
internal
|
|
|
printer
Package printer display in standard output simple info/success/error messages
|
Package printer display in standard output simple info/success/error messages |
|
screen
Package screen provides building blocks to represent progress bars.
|
Package screen provides building blocks to represent progress bars. |
|
pkg
|
|
|
acl/aclcore
Package aclcore manage what an API consumer is granted to do, and issue access tokens to represent it.
|
Package aclcore manage what an API consumer is granted to do, and issue access tokens to represent it. |
|
acl/catalogacl
Package catalogacl contains the logic to authorise access to catalog resources based on user requesting it.
|
Package catalogacl contains the logic to authorise access to catalog resources based on user requesting it. |
|
acl/catalogaclview
Package catalogaclview is a layer on top of catalog (same business model) which apply ACL rules before performing any action
|
Package catalogaclview is a layer on top of catalog (same business model) which apply ACL rules before performing any action |
|
archiveadapters/archivedynamo
Package archivedynamo extends catalogdynamo to add media locations to the main table
|
Package archivedynamo extends catalogdynamo to add media locations to the main table |
|
archiveadapters/s3store
Package s3store implements archive.StoreAdapter with AWS S3 backend
|
Package s3store implements archive.StoreAdapter with AWS S3 backend |
|
backup
Package backup is providing commands to inspect a file system (hard-drive, USB, Android, S3) and backup medias to a remote DPhoto storage.
|
Package backup is providing commands to inspect a file system (hard-drive, USB, Android, S3) and backup medias to a remote DPhoto storage. |
|
backupadapters/analysers/exif
Package exif parse image files to extract key details.
|
Package exif parse image files to extract key details. |
|
backupadapters/analysers/m2ts
Package m2ts is parsing M2TS files followings specs: https://en.wikipedia.org/wiki/MPEG_transport_stream https://en.wikipedia.org/wiki/Packetized_elementary_stream https://en.wikipedia.org/wiki/Program-specific_information
|
Package m2ts is parsing M2TS files followings specs: https://en.wikipedia.org/wiki/MPEG_transport_stream https://en.wikipedia.org/wiki/Packetized_elementary_stream https://en.wikipedia.org/wiki/Program-specific_information |
|
backupadapters/analysers/mp4
Package mp4 parse a MP4 file to retrieve creation date, length, and other details found in the stream.
|
Package mp4 parse a MP4 file to retrieve creation date, length, and other details found in the stream. |
|
backupadapters/filesystemvolume
Package filesystemvolume scans a local filesystem to find medias in it
|
Package filesystemvolume scans a local filesystem to find medias in it |
|
catalog
Package catalog provides tools to maintain an index of all medias that have been backed up.
|
Package catalog provides tools to maintain an index of all medias that have been backed up. |
|
catalogadapters/catalogarchivesync
Package catalogarchivesync is calling archive domain directly
|
Package catalogarchivesync is calling archive domain directly |
|
catalogadapters/catalogdynamo
Package catalogdynamo package store all the data in a single multi-tenant table:
|
Package catalogdynamo package store all the data in a single multi-tenant table: |
|
dns
Package dns expose functions to renew SSL certificates on AWS
|
Package dns expose functions to renew SSL certificates on AWS |
|
meta
Package meta expose information about the application like the version.
|
Package meta expose information about the application like the version. |
|
tools
|
|
Click to show internal directories.
Click to hide internal directories.