Squid
5a0e2e630b
Ship the Go/SQLite API and Web UI, Chrome/Brave capture addon, Docker Compose, Pangolin reverse-proxy support, and a user-crontab watchdog so the binary stays running without systemd.
156 lines
5.3 KiB
Markdown
156 lines
5.3 KiB
Markdown
# RCS — Reddit Comment Saver
|
||
|
||
Self-hostable Go backend + Chrome/Brave extension that auto-saves Reddit comments (and the thread’s top post) from `www.reddit.com` and `old.reddit.com`, with screenshots, GLOB search, and optional Imgur export.
|
||
|
||
## Quick local dev
|
||
|
||
### 1. Backend
|
||
|
||
```bash
|
||
git checkout dev
|
||
make backend
|
||
```
|
||
|
||
Opens `http://127.0.0.1:8080`. SQLite + screenshots live under `./data/`.
|
||
|
||
To drop local seed/test rows (keeps real Reddit captures):
|
||
|
||
```bash
|
||
make prune-dev-data
|
||
```
|
||
|
||
Optional env vars:
|
||
|
||
| Variable | Default | Purpose |
|
||
|---|---|---|
|
||
| `RCS_ADDR` | `127.0.0.1:8080` | Listen address |
|
||
| `RCS_DATA_DIR` | `data` | DB + screenshots directory |
|
||
| `RCS_DB_PATH` | `$RCS_DATA_DIR/rcs.db` | SQLite path |
|
||
| `RCS_API_KEY` | _(empty)_ | If set, require `X-API-Key` on API routes |
|
||
| `RCS_PUBLIC_BASE_URL` | _(empty)_ | Canonical public origin (e.g. `https://rcs.example.com`) |
|
||
| `RCS_TRUST_PROXY` | _(off)_ | Set `1`/`true` to honor `X-Forwarded-Proto` / `X-Forwarded-Host` |
|
||
| `IMGUR_CLIENT_ID` | _(empty)_ | Enables Imgur export in the Web UI |
|
||
|
||
### 2. Extension (Chrome / Brave)
|
||
|
||
1. Open `chrome://extensions` (or `brave://extensions`)
|
||
2. Enable **Developer mode**
|
||
3. **Load unpacked** → select the `extension/` folder
|
||
4. Click the RCS icon → enter backend URL `http://127.0.0.1:8080` (and API key if you set one)
|
||
5. Browse a Reddit thread — visible comments are captured automatically (deduped by comment id)
|
||
|
||
### 3. Search Web UI
|
||
|
||
Open `http://127.0.0.1:8080`:
|
||
|
||
- Filter by **username** (prefix match)
|
||
- Free text uses SQLite **GLOB** (`*` and `?`). Example: `*bla*bla*`
|
||
- Screenshot thumbnails appear when captured; click to enlarge (lightbox)
|
||
- Each result can **Export to Imgur** (needs `IMGUR_CLIENT_ID`). After export: **Open on Imgur** + **Copy URL**. The Imgur URL is stored so the same image is not re-uploaded.
|
||
|
||
## Docker / VM / LXC
|
||
|
||
```bash
|
||
docker compose up -d --build
|
||
```
|
||
|
||
Or build a binary:
|
||
|
||
```bash
|
||
make build # writes ./bin/rcs
|
||
RCS_ADDR=0.0.0.0:8080 RCS_DATA_DIR=/var/lib/rcs ./bin/rcs
|
||
```
|
||
|
||
Example systemd unit:
|
||
|
||
```ini
|
||
[Unit]
|
||
Description=RCS Reddit Comment Saver
|
||
After=network.target
|
||
|
||
[Service]
|
||
ExecStart=/usr/local/bin/rcs
|
||
Environment=RCS_ADDR=0.0.0.0:8080
|
||
Environment=RCS_DATA_DIR=/var/lib/rcs
|
||
Environment=RCS_API_KEY=
|
||
Environment=RCS_PUBLIC_BASE_URL=
|
||
Environment=RCS_TRUST_PROXY=
|
||
Environment=IMGUR_CLIENT_ID=
|
||
Restart=on-failure
|
||
|
||
[Install]
|
||
WantedBy=multi-user.target
|
||
```
|
||
|
||
### User crontab (watchdog)
|
||
|
||
Keeps `bin/rcs` running without systemd: every minute the watchdog checks a pidfile and starts the process if it is down.
|
||
|
||
```bash
|
||
make build
|
||
cp -n data/rcs.env.example data/rcs.env # if not created yet
|
||
# edit data/rcs.env (RCS_ADDR, Pangolin URL, API key, Imgur, …)
|
||
make install-cron
|
||
```
|
||
|
||
That installs:
|
||
|
||
```cron
|
||
# RCS watchdog
|
||
* * * * * /absolute/path/to/RCS/scripts/rcs-watchdog.sh
|
||
```
|
||
|
||
Logs append to `data/rcs.log`. Check status with `tail -f data/rcs.log` or `make test-api`. Remove with `make uninstall-cron`.
|
||
|
||
Cron does not rebuild Go — re-run `make build` after pulling updates. Override paths via `RCS_BIN`, `RCS_ENV_FILE`, etc. if needed (see [`scripts/rcs-watchdog.sh`](scripts/rcs-watchdog.sh)).
|
||
|
||
When the backend is not on localhost, grant the extension host permission when saving the URL in the popup (Brave/Chrome will prompt).
|
||
|
||
## Reverse proxy (Pangolin)
|
||
|
||
RCS is meant to sit privately behind a reverse proxy. Auth stays at the proxy — leave `RCS_API_KEY` empty when Pangolin alone is enough.
|
||
|
||
### Pangolin protected subdomain
|
||
|
||
1. Run RCS so only Pangolin/Newt can reach it (e.g. `RCS_ADDR=0.0.0.0:8080` on a private site).
|
||
2. Create a public resource such as `https://rcs.example.com` pointing at upstream `http://127.0.0.1:8080` (or the container IP).
|
||
3. Enable Pangolin authentication on that resource (protected / SSO).
|
||
4. Start RCS with:
|
||
|
||
```bash
|
||
RCS_ADDR=0.0.0.0:8080 \
|
||
RCS_PUBLIC_BASE_URL=https://rcs.example.com \
|
||
RCS_TRUST_PROXY=1 \
|
||
./bin/rcs
|
||
```
|
||
|
||
5. In the **same browser profile**, open `https://rcs.example.com` once and complete Pangolin login (sets the session cookie).
|
||
6. In the RCS extension popup, set Backend URL to `https://rcs.example.com` and grant host permission.
|
||
|
||
The extension sends cookies (`credentials: "include"`) on API calls, so Pangolin allows the request the same way the Web UI tab does. Other browser profiles / Incognito need their own login. If health checks fail with 401/403 or an HTML login page, open the Web UI, log in, and retry.
|
||
|
||
Serve RCS at the **subdomain root** (`/`), not a subpath — static assets and API paths are absolute from `/`.
|
||
|
||
## API (extension / integrations)
|
||
|
||
- `GET /api/health`
|
||
- `GET /api/comments/exists?id=t1_...` — `{ exists, has_screenshot }`
|
||
- `GET /api/posts/exists?id=t3_...`
|
||
- `POST /api/comments` — JSON body with text fields + optional `screenshot_base64` (data URL or raw base64)
|
||
- `POST /api/comments/{id}/screenshot` — attach screenshot to an existing comment (backfill)
|
||
- `POST /api/posts`
|
||
- `GET /api/search?q=&user=`
|
||
- `POST /api/comments/{id}/imgur`
|
||
- `GET /api/blocklist` — blocked usernames (synced to extensions)
|
||
- `POST /api/blocklist` — `{ "username": "AutoModerator" }`
|
||
- `DELETE /api/blocklist/{username}`
|
||
- `PUT /api/blocklist` — replace full list `{ "usernames": [...] }`
|
||
|
||
## Layout
|
||
|
||
```
|
||
backend/ Go server (SQLite, Web UI, Imgur)
|
||
extension/ Manifest V3 Chrome/Brave addon
|
||
data/ Local SQLite + screenshots (gitignored)
|
||
```
|