Commit Graph

9 Commits

Author SHA1 Message Date
ZhenYi
8fb2436f22 feat(git): add Redis-backed hook worker with per-repo distributed locking
- pool/worker.rs: single-threaded consumer that BLMPOPs from Redis queues
  sequentially. K8s replicas provide HA — each pod runs one worker.
- pool/redis.rs: RedisConsumer with BLMOVE atomic dequeue, ACK/NAK, and
  retry-with-json support.
- pool/types.rs: HookTask, TaskType, PoolConfig (minimal — no pool metrics).
- sync/lock.rs: Redis SET NX EX per-repo lock to prevent concurrent workers
  from processing the same repo. Lock conflicts are handled by requeueing
  without incrementing retry count.
- hook/mod.rs: HookService.start_worker() spawns the background worker.
- ssh/mod.rs / http/mod.rs: ReceiveSyncService RPUSHes to Redis queue.
  Both run_http and run_ssh call start_worker() to launch the consumer.
- Lock conflicts (GitError::Locked) in the worker are requeued without
  incrementing retry_count so another worker can pick them up.
2026-04-17 12:33:58 +08:00
ZhenYi
eeb99bf628 refactor(git): drop hook pool, sync execution is now direct and sequential
Some checks are pending
CI / Rust Lint & Check (push) Waiting to run
CI / Rust Tests (push) Waiting to run
CI / Frontend Lint & Type Check (push) Waiting to run
CI / Frontend Build (push) Blocked by required conditions
- Remove entire pool/ directory (RedisConsumer, CpuMonitor, LogStream, HookTask, TaskType)
- Remove Redis distributed lock (acquire_lock/release_lock) — K8s StatefulSet
  scheduling guarantees exclusive access per repo shard
- Remove sync/lock.rs, sync/remote.rs, sync/status.rs (dead code)
- Remove hook/event.rs (GitHookEvent was never used)
- New HookService exposes sync_repo / fsck_repo / gc_repo directly
- ReceiveSyncService now calls HookService inline instead of LPUSH to Redis queue
- sync/mod.rs: git2 operations wrapped in spawn_blocking for Send safety
  (git2 types are not Send — async git2 operations must not cross await points)
- scripts/push.js: drop 'frontend' from docker push list (embedded into static binary)
2026-04-17 12:22:09 +08:00
ZhenYi
3de4fff11d feat(service): improve model sync and harden git HTTP/SSH stability
Some checks are pending
CI / Rust Lint & Check (push) Waiting to run
CI / Rust Tests (push) Waiting to run
CI / Frontend Lint & Type Check (push) Waiting to run
CI / Frontend Build (push) Blocked by required conditions
Model sync:
- Filter OpenRouter models by what the user's AI client can actually access,
  before upserting metadata (avoids bloating with inaccessible models).
- Fall back to direct endpoint sync when no OpenRouter metadata matches
  (handles Bailian/MiniMax and other non-OpenRouter providers).

Git stability fixes:
- SSH: add 5s timeout on stdin flush/shutdown in channel_eof and
  cleanup_channel to prevent blocking the event loop on unresponsive git.
- SSH: remove dbg!() calls from production code paths.
- HTTP auth: pass proper Logger to SshAuthService instead of discarding
  all auth events to slog::Discard.

Dependencies:
- reqwest: add native-tls feature for HTTPS on Windows/Linux/macOS.
2026-04-17 00:13:40 +08:00
ZhenYi
0a998affbb refactor(git): remove SSH rate limiting
SSH is deployed inside Kubernetes cluster where rate limiting
at the application layer is unnecessary. Remove all SSH rate
limiter code:
- SshRateLimiter from SSHandle and SSHServer structs
- is_user_allowed checks in auth_publickey, auth_publickey_offered
- is_repo_access_allowed in exec_request
- is_ip_allowed in server::new_client
- rate_limiter module and start_cleanup
2026-04-16 22:40:59 +08:00
ZhenYi
bbf2d75fba fix(git): harden hook pool retry, standardize slog log format
Some checks are pending
CI / Rust Lint & Check (push) Waiting to run
CI / Rust Tests (push) Waiting to run
CI / Frontend Lint & Type Check (push) Waiting to run
CI / Frontend Build (push) Blocked by required conditions
- Add retry_count to HookTask with serde(default) for backwards compat
- Limit hook task retries to MAX_RETRIES=5, discard after limit to prevent
  infinite requeue loops that caused 'task nack'd and requeued' log spam
- Add nak_with_retry() in RedisConsumer to requeue with incremented count
- Standardize all slog logs: replace "info!(l, "msg"; "k" => v)" shorthand
  with "info!(l, "{}", format!("msg k={}", v))" across ssh/authz.rs,
  ssh/handle.rs, ssh/server.rs, hook/webhook_dispatch.rs, hook/pool/mod.rs
2026-04-16 21:41:35 +08:00
ZhenYi
02847ef1db fix(git): downgrade russh to 0.50.4, remove flate2 feature, fix log format
- Downgrade russh from 0.55.0 to 0.50.4
- Remove unused flate2 feature from russh dependency
- Use info!(logger, "{}", format!(...)) for channel lifecycle log messages
2026-04-16 20:58:01 +08:00
ZhenYi
1090359951 fix(git): add SSH channel lifecycle logging and fix password auth username check
- Remove user=="git" restriction from auth_password: the actual user is
  determined by the token, not the SSH username, matching Gitea's approach
- Add channel_open_session logging with explicit flush to verify
  CHANNEL_OPEN_CONFIRMATION reaches the client
- Add pty_request handler (reject with log) so git clients that request
  a PTY are handled gracefully instead of falling through to default
- Add subsystem_request handler (log + accept) so git subsystems are
  visible in logs
- Prefix unused variables with _ to eliminate warnings
2026-04-16 20:40:17 +08:00
ZhenYi
cef4ff1289 fix(git): harden HTTP and SSH git transports for robustness
HTTP:
- Return Err(...) instead of Ok(HttpResponse::...) for error cases so
  actix returns correct HTTP status codes instead of 200
- Add 30s timeout on info_refs and handle_git_rpc git subprocess calls
- Add 1MB pre-PACK limit to prevent memory exhaustion on receive-pack
- Enforce branch protection rules (forbid push/force-push/deletion/tag)
- Simplify graceful shutdown (remove manual signal handling)

SSH:
- Fix build_git_command: use block match arms so chained .arg() calls
  are on the Command, not the match expression's () result
- Add MAX_RETRIES=5 to forward() data-pump loop to prevent infinite
  spin on persistent network failures
- Fall back to raw path if canonicalize() fails instead of panicking
- Add platform-specific git config paths (/dev/null on unix, NUL on win)
- Start rate limiter cleanup background task so HashMap doesn't grow
  unbounded over time
- Derive Clone on RateLimiter so SshRateLimiter::start_cleanup works
2026-04-16 20:11:18 +08:00
ZhenYi
93cfff9738 init 2026-04-15 09:08:09 +08:00