plex

README

PLEX 🧫×🧬→💊

⚡ Build highly reproducible container workflows on top of a decentralised compute network. ⚡

PLEX is a simple client for distributed computation.

• 🌎 Build once, run anywhere at any scale: PLEX is using distributed compute and storage to run containers on a public network. Need GPUs? We got you covered.
• 🔗 Strictly composable: Every tool in PLEX has declared inputs and outputs. Plugging together tools by other authors should be easy.
• 🔍 Content-addressed by default: Every file processed by PLEX has a deterministic address based on its content. Keep track of your files and always share the right results with other scientists.

PLEX is based on Bacalhau, IPFS, and inspired by Pydantic and the Common Workflow Language.

🚀 PLEX in one minute

Install the client and adjust permissions.

source <(curl -sSL https://raw.githubusercontent.com/labdao/plex/main/install.sh)
chmod +x ./plex

Claim an access token and submit an example.

./plex -tool tools/equibind.json -input-dir ./testdata/binding/abl

💡 Use-Cases

• 🧬 run PLEX to design proteins with colabfold and RFDiffusion
• 💊 run PLEX to run small molecule docking with equibind and diffdock
• 🐋 configure your containerised tool to run on PLEX

🧑‍💻 Developer Guide

Building PLEX from source

git clone https://github.com/labdao/plex
cd plex
go build

Running a compute node

This is a script for setting up a compute instance to run LabDAO jobs. Requires linux OS with Nvidia GPU.

Tested on Ubuntu 20.04 LTS with Nvidia T4, V100, and A10 GPUs (AWS G4, P3, and G5 instance types)

The install script sets up Docker, Nvidia Drivers, Nvidia Container Toolkit, and IPFS

curl -sL https://raw.githubusercontent.com/labdao/plex/main/scripts/provide-compute.sh | bash && newgrp docker

After the script run the following command in a separate terminal to start a Bacalhau server to accept jobs.

ipfs daemon

Once the daemon is running, configure the Bacalhau node based on the addresses used by the IFPS node.

ipfs id

# copy the ip4 tcp output and change port 4001 to 5001 then export
export IPFS_CONNECT=/ip4/127.0.0.1/tcp/5001/p2p/<your id goes here>

# example: export IPFS_CONNECT=/ip4/127.0.0.1/tcp/5001/p2p/12D3KooWPH1BpPfNXwkf778GMP2H5z7pwjKVQFnA5NS3DngU7pxG

LOG_LEVEL=debug bacalhau serve --job-selection-accept-networked --limit-total-gpu 1 --limit-total-memory 12gb --ipfs-connect $IPFS_CONNECT

To download large bacalhau results the below command may need ran

sudo sysctl -w net.core.rmem_max=2500000

💁 Contributing

PRs are welcome! Please consider our Contribute Guidelines when joining.

From time to time, we also post help-wanted bounty issues - please consider our Bounty Policy when engaging with LabDAO.