Store and serve large media with Hyperblobs

Store large files and media as blobs on a Hypercore, replicate them over Hyperswarm, and serve them to a UI over local HTTP with range requests—the Pear/Bare logic, no UI required.

This guide focuses on the Pear/Bare logic. It shows hyperblobs and hypercore-blob-server on their own—no Electron, no UI. For the same blob plumbing wired into full desktop apps, see Stream stored video in a peer-to-peer app, Stream a live camera in a peer-to-peer app, and Back up photos in a peer-to-peer app. The blob logic is identical across all three; only the UI and the frame/file source change. To share a whole folder of files over Hyperdrive instead, see Share files in a peer-to-peer app.

A Hypercore is an append-only log of small blocks—great for messages, awkward for a 200 MB video. Hyperblobs solves that: it chunks arbitrarily large binary data across a Hypercore and hands back a small blob id that addresses it. hypercore-blob-server then serves any blob over local HTTP (127.0.0.1) so a <video> or <img> tag—or any HTTP client—can stream it with range requests, fetching only the bytes it needs.

This is purely Pear-end logic: it runs in a Bare worker (or any Bare/Node process) and never touches a UI.

This guide is about the Pear-end, not the shell. The code below lives in the Bare worker—the peer-to-peer logic, not the user interface. Because the Pear-end never imports DOM APIs and never assumes a UI framework, the same worker is portable across desktop (Electron), mobile (React Native via Bare iOS / Bare Android), and terminal. The example apps ship an Electron shell, but only the UI half changes per platform—the logic here stays the same. See Runtime and languages for the cross-platform model and current support.

How addressing works

Writing a blob returns an id describing where the bytes live in the core:

{ byteOffset, blockOffset, blockLength, byteLength }

Combine that id with the blobs core key and you have everything a peer needs to fetch the blob—directly as bytes, or through a blob-server link.

Add the dependencies

npm install corestore hyperswarm hyperblobs hypercore-blob-server hypercore-id-encoding

Write and seed a blob

The writer:

opens a Corestore namespace and wraps it with Hyperblobs (L12–L13)
announces the blobs core on Hyperswarm so readers can find it (L16)
streams a local file through createWriteStream() (L18–L24)
and prints the { key, ...id } payload other peers need (L27–L28):

writer-app/index.js

import Corestore from 'corestore'
import Hyperswarm from 'hyperswarm'
import Hyperblobs from 'hyperblobs'
import idEnc from 'hypercore-id-encoding'
import fs from 'bare-fs'
import process from 'bare-process'

const store = new Corestore('./writer-store')
const swarm = new Hyperswarm()
swarm.on('connection', (conn) => store.replicate(conn))

const blobs = new Hyperblobs(store.get({ name: 'blobs' }))
await blobs.ready()

// Announce the blobs core so readers can discover and replicate it.
swarm.join(blobs.core.discoveryKey, { server: true, client: false })

// Store a local file as a blob.
const ws = blobs.createWriteStream()
fs.createReadStream('./clip.mp4').pipe(ws)
await new Promise((resolve, reject) => {
  ws.on('error', reject)
  ws.on('close', resolve)
})

// Share this with readers — the core key plus the blob id.
const blob = { key: idEnc.normalize(blobs.core.key), ...ws.id }
console.log('blob:', JSON.stringify(blob))

// Keep seeding until interrupted, then tear down cleanly.
process.once('SIGINT', async () => {
  await blobs.close()
  await swarm.destroy()
  await store.close()
  process.exit(0)
})

Read or serve the blob on another peer

The reader:

parses that JSON blob descriptor from the command line (L9)
opens the remote core by key and replicates it (L15–L17)
Option A pulls the raw bytes with blobs.get() and the id fields (L20–L27)
Option B—what desktop apps use—starts hypercore-blob-server and prints a local link the renderer can pass to <video> or <img> (L30–L33)

Option B is what desktop apps use: the worker hands the link to the renderer, and the browser streams the media straight from hypercore-blob-server, requesting byte ranges as the user scrubs.

reader-app/index.js

import Corestore from 'corestore'
import Hyperswarm from 'hyperswarm'
import Hyperblobs from 'hyperblobs'
import BlobServer from 'hypercore-blob-server'
import idEnc from 'hypercore-id-encoding'
import process from 'bare-process'

// { key, byteOffset, blockOffset, blockLength, byteLength }
const blob = JSON.parse(process.argv[2])

const store = new Corestore('./reader-store')
const swarm = new Hyperswarm()
swarm.on('connection', (conn) => store.replicate(conn))

const core = store.get({ key: idEnc.decode(blob.key) })
await core.ready()
swarm.join(core.discoveryKey, { client: true, server: false })

// Option A — read the raw bytes.
const blobs = new Hyperblobs(core)
const bytes = await blobs.get({
  byteOffset: blob.byteOffset,
  blockOffset: blob.blockOffset,
  blockLength: blob.blockLength,
  byteLength: blob.byteLength
})
console.log('read bytes:', bytes.byteLength)

// Option B — serve over local HTTP so a <video>/<img> can stream it with range requests.
const server = new BlobServer(store.session())
await server.listen()
const link = server.getLink(blob.key, { blob, type: 'video/mp4' })
console.log('stream from:', link)

// Close the server and blobs before the swarm and store on exit.
process.once('SIGINT', async () => {
  await server.close()
  await blobs.close()
  await swarm.destroy()
  await store.close()
  process.exit(0)
})

Tear it down

Each app cleans up on Ctrl+C: the SIGINT handler at the end of every file closes the blobs (and, in the reader, the HTTP server) before the swarm and store, so in-flight reads and the local server shut down before the underlying core does.