Add self-hosted RCS backend, extension, and ops tooling
Ship the Go/SQLite API and Web UI, Chrome/Brave capture addon, Docker Compose, Pangolin reverse-proxy support, and a user-crontab watchdog so the binary stays running without systemd.
This commit is contained in:
@@ -1,3 +1,155 @@
|
||||
# RCS
|
||||
# RCS — Reddit Comment Saver
|
||||
|
||||
Reddit Comment Saved
|
||||
Self-hostable Go backend + Chrome/Brave extension that auto-saves Reddit comments (and the thread’s top post) from `www.reddit.com` and `old.reddit.com`, with screenshots, GLOB search, and optional Imgur export.
|
||||
|
||||
## Quick local dev
|
||||
|
||||
### 1. Backend
|
||||
|
||||
```bash
|
||||
git checkout dev
|
||||
make backend
|
||||
```
|
||||
|
||||
Opens `http://127.0.0.1:8080`. SQLite + screenshots live under `./data/`.
|
||||
|
||||
To drop local seed/test rows (keeps real Reddit captures):
|
||||
|
||||
```bash
|
||||
make prune-dev-data
|
||||
```
|
||||
|
||||
Optional env vars:
|
||||
|
||||
| Variable | Default | Purpose |
|
||||
|---|---|---|
|
||||
| `RCS_ADDR` | `127.0.0.1:8080` | Listen address |
|
||||
| `RCS_DATA_DIR` | `data` | DB + screenshots directory |
|
||||
| `RCS_DB_PATH` | `$RCS_DATA_DIR/rcs.db` | SQLite path |
|
||||
| `RCS_API_KEY` | _(empty)_ | If set, require `X-API-Key` on API routes |
|
||||
| `RCS_PUBLIC_BASE_URL` | _(empty)_ | Canonical public origin (e.g. `https://rcs.example.com`) |
|
||||
| `RCS_TRUST_PROXY` | _(off)_ | Set `1`/`true` to honor `X-Forwarded-Proto` / `X-Forwarded-Host` |
|
||||
| `IMGUR_CLIENT_ID` | _(empty)_ | Enables Imgur export in the Web UI |
|
||||
|
||||
### 2. Extension (Chrome / Brave)
|
||||
|
||||
1. Open `chrome://extensions` (or `brave://extensions`)
|
||||
2. Enable **Developer mode**
|
||||
3. **Load unpacked** → select the `extension/` folder
|
||||
4. Click the RCS icon → enter backend URL `http://127.0.0.1:8080` (and API key if you set one)
|
||||
5. Browse a Reddit thread — visible comments are captured automatically (deduped by comment id)
|
||||
|
||||
### 3. Search Web UI
|
||||
|
||||
Open `http://127.0.0.1:8080`:
|
||||
|
||||
- Filter by **username** (prefix match)
|
||||
- Free text uses SQLite **GLOB** (`*` and `?`). Example: `*bla*bla*`
|
||||
- Screenshot thumbnails appear when captured; click to enlarge (lightbox)
|
||||
- Each result can **Export to Imgur** (needs `IMGUR_CLIENT_ID`). After export: **Open on Imgur** + **Copy URL**. The Imgur URL is stored so the same image is not re-uploaded.
|
||||
|
||||
## Docker / VM / LXC
|
||||
|
||||
```bash
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
Or build a binary:
|
||||
|
||||
```bash
|
||||
make build # writes ./bin/rcs
|
||||
RCS_ADDR=0.0.0.0:8080 RCS_DATA_DIR=/var/lib/rcs ./bin/rcs
|
||||
```
|
||||
|
||||
Example systemd unit:
|
||||
|
||||
```ini
|
||||
[Unit]
|
||||
Description=RCS Reddit Comment Saver
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
ExecStart=/usr/local/bin/rcs
|
||||
Environment=RCS_ADDR=0.0.0.0:8080
|
||||
Environment=RCS_DATA_DIR=/var/lib/rcs
|
||||
Environment=RCS_API_KEY=
|
||||
Environment=RCS_PUBLIC_BASE_URL=
|
||||
Environment=RCS_TRUST_PROXY=
|
||||
Environment=IMGUR_CLIENT_ID=
|
||||
Restart=on-failure
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
|
||||
### User crontab (watchdog)
|
||||
|
||||
Keeps `bin/rcs` running without systemd: every minute the watchdog checks a pidfile and starts the process if it is down.
|
||||
|
||||
```bash
|
||||
make build
|
||||
cp -n data/rcs.env.example data/rcs.env # if not created yet
|
||||
# edit data/rcs.env (RCS_ADDR, Pangolin URL, API key, Imgur, …)
|
||||
make install-cron
|
||||
```
|
||||
|
||||
That installs:
|
||||
|
||||
```cron
|
||||
# RCS watchdog
|
||||
* * * * * /absolute/path/to/RCS/scripts/rcs-watchdog.sh
|
||||
```
|
||||
|
||||
Logs append to `data/rcs.log`. Check status with `tail -f data/rcs.log` or `make test-api`. Remove with `make uninstall-cron`.
|
||||
|
||||
Cron does not rebuild Go — re-run `make build` after pulling updates. Override paths via `RCS_BIN`, `RCS_ENV_FILE`, etc. if needed (see [`scripts/rcs-watchdog.sh`](scripts/rcs-watchdog.sh)).
|
||||
|
||||
When the backend is not on localhost, grant the extension host permission when saving the URL in the popup (Brave/Chrome will prompt).
|
||||
|
||||
## Reverse proxy (Pangolin)
|
||||
|
||||
RCS is meant to sit privately behind a reverse proxy. Auth stays at the proxy — leave `RCS_API_KEY` empty when Pangolin alone is enough.
|
||||
|
||||
### Pangolin protected subdomain
|
||||
|
||||
1. Run RCS so only Pangolin/Newt can reach it (e.g. `RCS_ADDR=0.0.0.0:8080` on a private site).
|
||||
2. Create a public resource such as `https://rcs.example.com` pointing at upstream `http://127.0.0.1:8080` (or the container IP).
|
||||
3. Enable Pangolin authentication on that resource (protected / SSO).
|
||||
4. Start RCS with:
|
||||
|
||||
```bash
|
||||
RCS_ADDR=0.0.0.0:8080 \
|
||||
RCS_PUBLIC_BASE_URL=https://rcs.example.com \
|
||||
RCS_TRUST_PROXY=1 \
|
||||
./bin/rcs
|
||||
```
|
||||
|
||||
5. In the **same browser profile**, open `https://rcs.example.com` once and complete Pangolin login (sets the session cookie).
|
||||
6. In the RCS extension popup, set Backend URL to `https://rcs.example.com` and grant host permission.
|
||||
|
||||
The extension sends cookies (`credentials: "include"`) on API calls, so Pangolin allows the request the same way the Web UI tab does. Other browser profiles / Incognito need their own login. If health checks fail with 401/403 or an HTML login page, open the Web UI, log in, and retry.
|
||||
|
||||
Serve RCS at the **subdomain root** (`/`), not a subpath — static assets and API paths are absolute from `/`.
|
||||
|
||||
## API (extension / integrations)
|
||||
|
||||
- `GET /api/health`
|
||||
- `GET /api/comments/exists?id=t1_...` — `{ exists, has_screenshot }`
|
||||
- `GET /api/posts/exists?id=t3_...`
|
||||
- `POST /api/comments` — JSON body with text fields + optional `screenshot_base64` (data URL or raw base64)
|
||||
- `POST /api/comments/{id}/screenshot` — attach screenshot to an existing comment (backfill)
|
||||
- `POST /api/posts`
|
||||
- `GET /api/search?q=&user=`
|
||||
- `POST /api/comments/{id}/imgur`
|
||||
- `GET /api/blocklist` — blocked usernames (synced to extensions)
|
||||
- `POST /api/blocklist` — `{ "username": "AutoModerator" }`
|
||||
- `DELETE /api/blocklist/{username}`
|
||||
- `PUT /api/blocklist` — replace full list `{ "usernames": [...] }`
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
backend/ Go server (SQLite, Web UI, Imgur)
|
||||
extension/ Manifest V3 Chrome/Brave addon
|
||||
data/ Local SQLite + screenshots (gitignored)
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user