RIHNO Framework 🦏: Distributed Cloud & P2P Storage Engine

Overview

RIHNO Framework is an advanced distributed storage system built in Go (1.25) featuring two tightly integrated subsystems that share a unified on-disk storage and cryptographic engine:

1. Peer-to-Peer Content-Addressable Storage (CAS): Nodes communicate over raw TCP, storing file chunks on local disk using a SHA-1 content-addressable path hierarchy, and encrypting every byte with AES-256-CTR before disk writes or network transmission. Nodes without a local file query peers across the mesh network and stream data back on demand.
2. Google Drive Pooling Engine & Web Portal: Pools multiple free Google Drive accounts (15GB each) into one virtually infinite storage volume. Files are compressed with DEFLATE, encrypted client-side, split into chunks, and uploaded concurrently to whichever registered account has the most free space. All chunks and file manifests are tracked in a shared PostgreSQL database (e.g. Neon) and accessible through a Cobra CLI and a multi-user Vue 3 web portal.

🎯 Key Features

🌐 Peer-to-Peer Content-Addressable Storage

• Raw TCP Transport: Custom framing protocol (IncomingMessage vs IncomingStream) separating discrete RPC control messages from high-throughput raw file streams.
• SHA-1 Path Sharding: Prevents flat directory bottleneck by sharding hash digests into 5-character nested subdirectories (e.g. 68044/29f74/181a6/...).
• P2P Mesh Discovery & Replication: Nodes broadcast MessageStoreFile to connected peers for automatic replication and fetch missing files via MessageGetFile.

☁️ Google Drive Multi-Account Pooling

• Virtual Unified Storage: Pools $N$ registered 15GB Google Drive accounts into a unified virtual disk, eliminating single-account file size limits.
• Smart Chunk Placement: Concurrent multi-part uploads placed dynamically on whichever account currently has the greatest available capacity.
• Selective DEFLATE Compression: Automatically compresses chunks with DEFLATE, retaining compressed data only if it yields genuine space savings while bypassing already-compressed media.
• AES-256-CTR Streaming Crypto: Stream-encrypts file bytes with AES-256 in Counter (CTR) mode with a fresh, random Initialization Vector (IV) prepended per chunk.

🖥️ Multi-User Web Portal & Modern UI

• Vue 3 + Vite SPA: Drive-like user interface with full-page drag-and-drop file uploads, search, and storage consumption bars per connected Drive account.
• Single Binary Distribution: The built Vue 3 frontend is embedded directly into the Go binary (go:embed), enabling self-contained deployments with zero runtime Node.js dependency.
• Multi-Tenant Isolation: Multi-user authentication using Gin, HTTP sessions, and GORM with scoped user tenant isolation.

🛠️ Powerful Cobra CLI

• Comprehensive command-line suite: rhino account add/list/remove, rhino put, rhino get, rhino ls, rhino status, and rhino serve.
• Automated OAuth2 loopback redirect consent flow for seamless Google account linking.

⚙️ Architecture & Subsystems

rhino-framework/
├── main.go                     # Entry point for bare TCP listener
├── server.go                   # FileServer: P2P store/get API, peer map, broadcast
├── storage/                    # CAS on-disk storage & crypto engine
│   ├── store.go                # Store, CASPathTransformFunc (SHA-1 path sharding)
│   └── crypto.go               # AES-256-CTR streaming crypto, MD5 key hashing
├── p2p/                        # Transport & networking layer
│   ├── transport.go            # Transport & Peer interfaces
│   ├── tcp_transport.go        # TCPTransport (dial/accept loops, stream handling)
│   └── message.go              # RPC struct + framing byte constants
├── drivepool/                  # Google Drive pooling domain logic
│   ├── pool.go                 # Pool / Account: PutStream/GetStream concurrent engine
│   ├── placement.go            # Dynamic quota tracker & chunk placement
│   ├── manifest/               # GORM models (Account, VirtualFile, Chunk)
│   └── gdrive/                 # Google Drive API v3 client wrapper
├── db/                         # Shared PostgreSQL connection & migrations (Neon)
│   ├── db.go                   # Pooled *gorm.DB connection
│   └── migrate.go              # Idempotent SQL migration runner
├── backend/                    # Gin-based REST API & SPA embedding
│   ├── server.go               # Gin Engine, routes & graceful shutdown
│   ├── handlers.go             # File & account CRUD handlers
│   └── dist/                   # Embedded Vue 3 static distribution
├── frontend/                   # Vue 3 + Vite SPA dashboard
│   ├── src/                    # Components, views, Pinia stores
│   └── tests/                  # Vitest test suite
└── cmd/rhino/                  # Cobra CLI binary & `rhino serve` entrypoint

🔐 Security & Cryptography

• At-Rest Token Encryption: All Google OAuth refresh tokens are encrypted at rest in PostgreSQL with a 32-byte master key (RHINO_TOKEN_ENCRYPTION_KEY).
• Client-Side Chunk Encryption: File chunks are encrypted locally prior to transmission; raw file data is never visible to Google Drive or intermediate network peers.
• Key Obfuscation: Storage keys sent over the P2P wire are hashed via MD5 to prevent plaintext filename inspection by network eavesdroppers.

🛠️ CLI Usage Guide

# Set user identity
export RHINO_USER=souvik

# Connect Google Drive accounts via browser OAuth loopback
bin/rhino account add --label personal-drive
bin/rhino account add --label secondary-drive

# List connected accounts and free capacity
bin/rhino account list
# Output:
# personal-drive   14.8 GiB free of 15.0 GiB
# secondary-drive  13.2 GiB free of 15.0 GiB

# Upload a file across pooled accounts
bin/rhino put ~/videos/demo.mp4 --as videos/demo.mp4

# Check pooled virtual drive status
bin/rhino status
# Output: 2/2 accounts healthy | 30.0 GiB total | 28.0 GiB free | 1 files

# Retrieve and reconstruct file
bin/rhino get videos/demo.mp4 ~/restored_demo.mp4

🤸 Quick Start & Local Setup

1. Prerequisites

• Go: Version 1.25 or newer
• PostgreSQL: Neon DB connection string or local Postgres 14+
• Google Cloud OAuth: Google Cloud Console project with Google Drive API enabled and OAuth client JSON (client_secret.json)

2. Clone & Build CLI

git clone https://github.com/Souvik-223/rhino-merge-framework.git
cd rhino-merge-framework
go mod download

# Build binaries
go build -o bin/fs .
go build -o bin/rhino ./cmd/rhino

3. Run the Web Portal

# Build frontend and compile embedded portal binary
make build-frontend
make build-portal

# Start portal server
./bin/rhino serve

4. Deploying with Docker Compose & Caddy

cp .env.example .env
mkdir -p secrets

# Generate security keys
echo -n "$(openssl rand -base64 32)" > secrets/session_secret.txt
echo -n "postgres://user:pass@host/db?sslmode=require" > secrets/database_url.txt
echo -n "$(openssl rand -hex 32)" > secrets/token_encryption_key.txt
cp client_secret.json secrets/client_secret.json

# Launch containerized stack
docker compose up -d --build

🧪 Comprehensive Automated Testing

The entire P2P and Drive pooling test suites run against in-memory mock stores and transactional PostgreSQL rollbacks without requiring real Google Cloud credentials:

# Run all Go unit and integration tests
export DATABASE_URL="postgres://postgres:postgres@localhost:5432/rhino_dev?sslmode=disable"
go test ./... -v

# Run frontend Vitest test suite
cd frontend
npm run test