Reconnect options (-reconnect, -reconnect_streamed, etc.) only apply to
network inputs and cause 'Option reconnect not found' errors when the
input is pipe:0. Curl already handles connection resilience.
The previous -max_alloc approach does not fix the 'overlong headers' error
because FFmpeg 5.1 (Debian bookworm) uses a compile-time stack-allocated
buffer (MAX_URL_SIZE=4096) for HTTP response headers, which -max_alloc
cannot change.
Instead, when the stream URL is HTTP/HTTPS, launch curl to handle the
HTTP connection and pipe its stdout to FFmpeg via stdin (pipe:0). Curl
does not have the 4096-byte header limit, so it handles oversized CDN
response headers from Douyin without error.
Additional changes:
- RequestStopAsync kills curl first (instead of sending 'q' to FFmpeg),
causing the pipe to close and FFmpeg to exit gracefully on EOF.
- SessionProcessRuntime tracks the curl process for cleanup.
- Add -max_alloc 100000000 to FFmpeg arguments for HTTP inputs to avoid
'overlong headers' error when CDN (e.g. Douyin) returns oversized HTTP
response headers exceeding FFmpeg's default 4096-byte buffer.
- Add exponential backoff for repeated startup failures (30s → 15min cap)
to break the tight fail→retry→re-poll loop that floods notifications.
- Throttle startup failure notifications to at most one per 30 minutes per
room to prevent email/webhook storms during persistent failures.
- Reset backoff counter when a session successfully opens its first segment.
The ternary in GetLowStoragePauseMessageAsync had its branches swapped:
when storage was healthy it returned the pause error, and when storage
was critically low it returned null (allowing finalization to proceed).
This caused MP4 finalization to always pause with a misleading 'storage
is Red' warning even when storage protection was disabled.
API and nginx services now pull from reg.nxsir.cn/live_recorder
instead of building locally. The 'build' sections referenced the old
Dockerfiles with WSL-specific binfmt/proxy workarounds; the CI pipeline
in Jenkinsfile now handles multi-arch builds and pushes.
Uses patchable REGISTRY_URL env var (defaults to reg.nxsir.cn).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The frontend Dockerfile was missing HTTP_PROXY/HTTPS_PROXY ARG declarations
in the build stage, so npm ci could not reach the npm registry through the
builder's proxy. Same fix pattern as the API Dockerfile.
This explains why the Web image has never been successfully built — the
pipeline always skips 'Build Web' after 'Build API' fails, but once API
build succeeds, Web build would hit the same npm registry issue.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The case statement that sets RID based on TARGETARCH runs in a /bin/sh
subshell. When it was in a separate RUN from dotnet restore/publish,
the RID variable was not available to the publish command.
Merge the case dispatch, dotnet restore, and dotnet publish into a
single RUN so the RID shell variable stays in scope. Also restores
the COPY . . directive that was accidentally dropped.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The previous inline case statement was missing proper variable assignment.
Use 'case' to set DOTNET_RID directly instead of trying to capture output
of a command substitution that contained a multi-branch case.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Removing RuntimeIdentifier from the csproj caused dotnet publish to produce
ALL platform runtimes (20+) in the output, making runtime loading ambiguous
and causing FileLoadException at startup.
Instead of hardcoding the RID in the csproj, use Docker's built-in TARGETARCH
ARG (injected automatically by buildx for multi-platform builds):
amd64 -> -r linux-x64
arm64 -> -r linux-arm64
This ensures native deps (SQLitePCLRaw, EF Core) resolve to exactly one
architecture per image, while keeping the csproj clean for local dev.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
docker driver only supports a single instance. Creating another fails with
'additional instances of driver docker cannot be created'. Just use the
default builder that ships with the docker daemon.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The docker-container driver requires moby/buildkit:buildx-stable-1 from
Docker Hub, which is unreachable from the build network (EOF / timeout).
The docker driver runs buildkit inside the host daemon without needing
a separate container image.
Also explicitly rm + recreate the builder each run so stale
docker-container instances don't linger as the default.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Removing <RuntimeIdentifier> from the csproj changed the publish layout,
but the old registry build cache still contains the RID-poisoned layers.
Subsequent builds cached at the COPY/RUN layer boundary reuse those stale
layers, causing FileLoadException at runtime.
Add --no-cache to both API and Web buildx invocations so every build
produces fresh layers. Once the pipeline stabilizes, we can re-enable
cache-from with the updated cache tag.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
:latest manifests already exist from the first successful run (build 123),
so subsequent docker manifest create without --amend fails with
'refusing to amend an existing manifest list'. Add --amend so every
pipeline run can update the :latest tag.
Also remove DEBUG config.json prints now that the auth path is verified.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The csproj had <RuntimeIdentifier>linux-arm64</RuntimeIdentifier> baked in,
which forced dotnet publish to produce linux-arm64 native output on EVERY
platform (including amd64). On amd64 containers the runtime could not load
the entry assembly, causing FileLoadException crash loops.
RID is unnecessary for framework-dependent builds (--p:UseAppHost=false
is already in the Dockerfile). Removing it lets each platform produce
its natural runtime output — the base dotnet/aspnet image already has
the correct architecture's runtime.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
nuget.org is only reachable through the 7890 proxy from the build network
(direct hits return a 302 redirect loop; proxied requests return 200,
verified 8/8). The pipeline passes --build-arg HTTP_PROXY/HTTPS_PROXY, but
Dockerfile did not declare these ARGs in the build stage, so buildkit did
not inject them into the RUN environment and 'dotnet restore' tried nuget.org
directly -> NU1301 'Unable to load the service index'.
Declare HTTP_PROXY/HTTPS_PROXY/NO_PROXY (upper+lower case) as build-stage
ARGs and promote them to ENV so dotnet's HttpClient uses the proxy. Also add
two restore retries (--disable-parallel) to smooth over transient proxy
blips.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Harbor project is named 'live_recorder' (with underscore), matching
the robot account robot$live_recorder+live. The pipeline was pushing to
'liverecorder' (no underscore) — a different/nonexistent project path —
so the robot's push permission did not apply and every push got 401
despite 'Login Succeeded'.
This was THE root cause of the persistent 401s, not buildx auth
forwarding or token expiry (those were all red herrings).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Every variant of docker login (--password-stdin, -p) has shown
'Login Succeeded' but docker push consistently gets 401.
Write the base64-encoded user:pass directly into
/root/.docker/config.json, eliminating docker login as a
variable entirely. If push still 401s after this, the problem
is definitively on the Harbor side (robot permissions).
Also fixes broken heredoc escaping from previous commit where
<<'DOCKERCFG' prevented shell variable expansion of AUTH_B64.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Suspect that echo + pipe to --password-stdin may be mangling the Jenkins
masked password variable. Switch to -p (command-line password) for a
cleaner auth path and add debug output to inspect /root/.docker/config.json
after login.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
docker push is also getting 401 (not just buildx --push), suggesting
the Harbor login token expires during long builds or there is a
credential propagation gap between the standalone Login stage and the
build stages.
Move withCredentials into each build stage and re-login immediately
before every docker push / manifest push. This gives each push
operation a fresh token.
Pipeline is now: Checkout -> Prepare Buildx -> Build API (login + build
+ push + manifest) -> Build Web (same).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
buildx build --push has failed 401 on every attempt. The buildkit auth
forwarding (whether via docker-container or docker driver) does not work
reliably on this builder.
New strategy: build each platform separately with --load (into local
docker, which can read /root/.docker/config.json), then push with
native docker push, then assemble a multi-arch manifest with
docker manifest create/push.
Per-platform tags are pushed as :<BUILD_ID>-amd64 / :<BUILD_ID>-arm64
and the manifest combines them under the canonical :<BUILD_ID> and :latest.
This replaces a single buildx --push call with:
1. buildx build --platform linux/amd64 --load
2. docker push (amd64)
3. docker rmi (free disk)
4. buildx build --platform linux/arm64 --load
5. docker push (arm64)
6. docker rmi (free disk)
7. docker manifest create + push (multi-arch)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The docker-container driver runs buildkit in a separate container that
cannot reliably forward host Docker registry credentials via the buildx
session mechanism, causing every --push to fail with 401 Unauthorized.
The --auth flag doesn't exist in buildx 0.23.0 on this builder.
Fix: switch to 'docker' driver which runs buildkit inside the host
Docker daemon and naturally shares its registry auth state.
Changes:
- Login Registry stage restored (before Prepare Buildx)
- Prepare Buildx: driver docker (not docker-container), no driver-opts
- Build stages: stripped withCredentials wrappers and --auth flags
- Removed buildkitd.toml max-parallelism config (docker driver doesn't
support it; swap provides the safety net for OOM)
Pipeline flow: Checkout -> Login -> Prepare Buildx -> Build API -> Build Web
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The docker-container driver's buildkit session mechanism does not reliably
forward host Docker credentials to the buildkit container, causing every
docker buildx build --push to fail with 401 Unauthorized on the manifest
HEAD request.
Fix: remove the standalone Login Registry stage and embed withCredentials
directly into each build stage, passing credentials to buildkit via the
docker buildx build --auth flag:
--auth 'reg.nxsir.cn=:'
This sends auth directly to buildkit rather than relying on the implicit
docker login -> config.json -> session forwarding chain.
Each stage also does a docker login for the host CLI (needed for
buildx inspect --bootstrap to pull images from the registry, and for
cache-from/cache-to operations).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Docker buildx create --driver-opt splits values on commas as list
separators, causing 'env.NO_PROXY=127.0.0.1,localhost,...' to be parsed
as separate key=value entries and failing with:
invalid value "localhost", expecting k=v
The NO_PROXY hosts for build containers are already passed via build-arg
in the build stage; the buildkit container itself doesn't need them.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move Login Registry stage before Prepare Buildx so Docker credentials are
in /root/.docker/config.json BEFORE the buildx docker-container builder
is created and bootstrapped. Previously the builder started without auth,
causing buildx --push to fail with '401 Unauthorized' on the manifest
HEAD request (blob layers pushed but manifest rejected).
Also expand the NO_PROXY driver-opt from just 'reg.nxsir.cn' to the full
NO_PROXY_HOSTS list (Tsighua, MCR, daocloud) so the buildkit container
itself also bypasses the proxy for these direct-reachable registries.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 7890 proxy intermittently returns '502 Bad Gateway' when tunneling to
mirrors.tuna.tsinghua.edu.cn (observed on libglapi-mesa during
apt-get install in the runtime layer). The Tsinghua mirror, MCR, and
daocloud are all directly reachable from this builder, so the proxy adds
no value and only introduces failure modes.
Add the following hosts to NO_PROXY_HOSTS so apt/curl inside the build
containers hit them directly:
- mirrors.tuna.tsinghua.edu.cn, .tsinghua.edu.cn
- mcr.microsoft.com
- docker.m.daocloud.io
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Multi-platform buildx (linux/amd64 + linux/arm64) races both platforms in
parallel by default. On the 4GB build agent, QEMU-emulated 'dotnet restore'
for arm64 alone spikes to 2-3GB and racing amd64 apt-installs push us into
the OOM Killer ("cannot allocate memory" at build 4/5).
Fix: write a small buildkitd.toml with max-parallelism = 1 and pass it to
'docker buildx create --config'. buildkit now runs stages sequentially so
the two platforms don't step on each other's RSS.
Recreate the builder every run so the config always takes effect.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The build agent cannot reach auth.docker.io from behind the local network,
causing 'docker run tonistiigi/binfmt --install arm64' to fail with a token
fetch timeout.
Fix: three-tier fallback for ARM64 binfmt registration:
1. Skip if /proc/sys/fs/binfmt_misc/qemu-aarch64 already exists.
2. Pull tonistiigi/binfmt via docker.m.daocloud.io mirror.
3. Fall back to qemu-user-static via apt.
Add a post-registration sanity check (arm64 alpine + uname -m) so
misconfigured builders fail fast rather than at buildx invocation time.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
DashboardService was constructing 'today in Beijing' bounds with a +08:00 offset, which Npgsql rejects when binding to a timestamp with time zone column. Added .ToUniversalTime() to normalize to UTC (same instant, offset 0), matching the existing pattern in SessionAnalyticsService.GetUtcWindow.
Under the Red storage tier, MP4 finalization (TS->MP4 remux) was being skipped, so tasks never reached Completed and the segment_completed event script — which uploads the file and deletes the local source to free space — never ran. The disk could never recover, deadlocking all recording and transcoding.
Two reversed checks caused this: (1) FfmpegService gated finalization on the legacy HasEnoughSpace MB threshold (effectively 4GB) instead of the tier system, and (2) the polling loop only resumed paused finalizations when NOT in the Red tier. Now finalization is gated solely on ShouldPauseActive (true Red only) and the polling loop always attempts to resume it every cycle, since finalization is the very mechanism that frees space. Once any segment finalizes, the upload+delete script runs and the disk recovers, letting the rest finish.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
scripts/build-arm64-image.sh automates the full linux/arm64 image build on Docker Desktop (WSL2): register qemu-aarch64 binfmt with the F flag so emulation works inside build containers, pre-pull the dotnet base images with resumable retries, build with dotnet restore routed through the host proxy and apt via the in-Dockerfile mirror, then docker save + gzip into a loadable archive. .gitignore now excludes the *.tar/*.tar.gz build artifacts, the qemu-*-static emulator binary, and the stray root package-lock.json.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Alias the replay player's danmakuEvents to replayDanmakuEvents so it no longer shadows another binding in the session detail view.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pin the WebApi to RuntimeIdentifier=linux-arm64 (framework-dependent) and rework the Dockerfile for ARM64: copy the full context up front, add QEMU emulation workarounds, and drop debug symbols / cap parallelism during publish.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Introduce a process-wide DatabaseCircuitBreaker that fails fast when Postgres is unavailable (e.g. disk full) instead of letting every request burn doomed EF Core retries. CircuitAwareExecutionStrategy derives from NpgsqlRetryingExecutionStrategy and records success/failure around the public Execute/ExecuteAsync seams; background workers skip work and back off while the circuit is open; the exception middleware maps an open circuit (and other DB outages) to 503. Adds /health (liveness) and /health/ready (readiness, reporting circuit state), plus unit tests for the open/half-open transitions and non-transient SQLSTATE detection.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a segment-level cleanup alongside the existing session-level one. DeleteMissingFileTasksAsync scans all non-active record tasks, keeps those whose video file no longer exists on disk, and deletes them individually via DeleteTasksAsync (which also drops any session left empty). Exposed as POST /record-tasks/delete-missing-files and a new "清理无文件分片" action in the record tasks view.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- CanStartNewRecording now purely based on Tier==Green (was HasEnoughSpace||Green)
- PollingBackgroundService now uses ShouldPauseActive instead of MB-based CheckShouldPause
- Both pause and start checks consolidated into single guardCheck call
- Old MB pause/resume thresholds still work as secondary safety via hasEnoughSpace
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
Add StorageGreenThresholdPercent (default 30%) and StorageRedThresholdPercent
(default 10%) to both SystemSettingsDto and the settings UI.
StorageGuardService now reads thresholds from settings instead of hardcoding.
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
DbContext is not thread-safe. Task.WhenAll caused concurrent access
within the same scoped DbContext, throwing 'A second operation was
started on this context instance' errors on slower machines.
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
VITE_API_BASE_URL may not be properly resolved during Docker build,
causing axios to construct malformed URLs.
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
Older Chromium on ARM64 doesn't support strict mode arguments.callee
in modern ES module bundles. Downgrading build target to es2015.
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
Raspberry Pi PostgreSQL is I/O constrained. 60s timeout was too short
for concurrent writes during heavy recording sessions, causing timeout +
retry causing duplicate key violations.
- CommandTimeout: 60s -> 120s
- maxRetryCount: 5 -> 3
- maxRetryDelay: 10s -> 15s
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
PlatformHttpClientFactory was holding a direct ISystemSettingsService reference
(Scoped). When the danmaku connection's request scope ended, retry attempts failed
with ObjectDisposedException on LiveRecorderDbContext.
Changed to use IServiceScopeFactory to create a fresh scope on each CreateAsync call,
so the danmaku retry loop always gets a live DbContext.
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
Storage Tier System:
- Add StorageTier enum (Green >30% / Yellow 10-30% / Red <10%)
- Extend StorageGuardResult with Tier, CanStartNewRecording, ShouldPauseActive, UsagePercent
- Yellow tier: deny new recordings but allow existing to finish and upload
- Red tier: deny new recordings and pause active sessions
- Auto-recovery: when disk frees up, polling automatically resumes new recordings
- Update LiveRoomPollingBackgroundService to use tier-based checks
- Expose tier + usage percent in Recovery API
Dashboard Queue Monitor:
- Add pending transcode count, pending upload count, queued data volume to dashboard
- Add storage tier badge (Green/Yellow/Red) with usage percentage
- Add queue monitoring card to dashboard view
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
Dashboard:
- Add DashboardDto, DashboardService with SQL-level aggregate queries
- Add GET /api/dashboard endpoint with real-time system status
- Add repository aggregate methods (CountByAvailability, SumDuration, etc.)
- Add DashboardView.vue as new landing page with KPI cards, storage status, recent sessions, top rooms
- Update router to make dashboard the new / route, add nav item in sidebar
File Preview:
- Add IVideoMetadataService + FfmpegVideoMetadataService for video metadata extraction
- Extend MediaBrowserItemDto with Metadata and ThumbnailUrl fields
- Add includeMetadata param to media browser API, add thumbnail endpoint
- Add SessionPlaylistDto and GET /api/record-sessions/{id}/playlist for continuous playback
Bandwidth Statistics:
- Add -progress pipe:1 to live recording ffmpeg args for bitrate output
- Parse total_size/bitrate/speed from ffmpeg progress lines during recording
- Write bandwidth samples as SystemLogEntry (Category=Bandwidth) every 30s
- Add BandwidthStatisticsService, BandwidthController with session timeline + daily summary
- Add bandwidth TypeScript types
Co-Authored-By: Claude Opus 4.8 noreply@anthropic.com
- Add IDanmakuService interface and DanmakuService implementation to parse danmaku XML files
- Add GET /api/record-tasks/{id}/danmaku and GET /api/record-sessions/{id}/danmaku endpoints
- Add DanmakuPlayer Vue component with native video + CSS overlay danmaku rendering
- Add danmakuEngine.ts pure-TypeScript animation loop with binary search, track management, and event notifications
- Add useDanmakuPlayer composable for reusable danmaku data loading
- Integrate danmaku toggle button into RecordTaskDetailView
- Integrate danmaku replay modal dialog into RecordSessionDetailView segment table
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
nanxun
