Commit Graph

16 Commits

Author SHA1 Message Date
ZhenYi
c7cee8c344 misc: polish git hooks, billing services, fctool, and API/WebSocket
- git: clean up hook pool worker, commit sync, HTTP rate limiting
- billing: tighten workspace/project/agent billing logic
- fctool: add project boards and issues management tools
- api/ws: minor room WebSocket protocol adjustments
- frontend: add RoomSettingsPanel component
2026-04-30 19:16:57 +08:00
ZhenYi
2a9ec6d509 feat(tag): vectorize repo tags after hook sync with incremental embedding + FC tool
- HookWorker gains optional embed_service field
- Captures changed tag names during webhook dispatch, batch-embeds after completion
- HookService auto-inits EmbedService from config for standalone git-hook binary
- Adds agent dep to git crate (no circular dep)
- SSH/HTTP servers no longer call start_worker (dedicated git-hook handles it)
- git_tag_search FC tool for agent semantic tag search with project isolation
2026-04-28 13:04:10 +08:00
ZhenYi
52a0131b56 fix(git): LFS token validation and remove IP rate limiting
- Implement proper token validation via user_token table (SHA256+base64 hash)
- Query token_hash, check IsRevoked, validate expiry
- Remove IP-based rate limiting (handled by k8s ingress)
- Remove unused client_ip() helper function
- user_uid() now async and queries database for real user
2026-04-27 16:40:01 +08:00
ZhenYi
bdb5393835 fix: resolve 30+ bugs from security audit
Critical:
- CORS: replace allow_any_origin + credentials with env-configured origins
- XSS: escape HTML before dangerouslySetInnerHTML in search results
- Path traversal: sanitize storage keys to reject ".." components
- Auth missing: add Session requirement to git init/open/is-repo endpoints
- Transaction: wrap issue cascade delete in DB transaction

High:
- Mutex poisoning: replace unwrap() with poison-recovering guards
- Drop tokio::spawn: use runtime handle or fallback thread for lock release
- Redis KEYS: replace with non-blocking SCAN for typing events
- SSH panic: handle missing stdin/stdout/stderr gracefully
- LFS auth: remove x-user-uid header injection vector, generate per-request tokens

Medium:
- Memory leak: remove Box::leak in provider normalization
- Race conditions: query closed count directly instead of subtraction
- Silent failures: add tracing::warn for AI tasks, room events, activity logs
- Frontend nav: sync activeRoomId when initialRoomId prop changes
- Duplicate nav: remove redundant setActiveRoom in delete handler
- Callback conflict: skip undefined values in updateCallbacks merge
- Stale closure: use wsClient state instead of wsClientRef.current in useMemo

Low:
- Captcha: validate captcha not empty before login submission
- Broadcast capacity: reduce from 100K to 1000
- Error handling: add try/catch for removeMember and updateMemberRole
- Loading state: show placeholder instead of null in RepositoryContextProvider
- WebSocket: add heartbeat ping and jitter to reconnect backoff
2026-04-27 10:57:23 +08:00
ZhenYi
64dc27161b chore(git): minor fixes and improvements across git library modules
Apply small fixes across multiple git ops files: handle errors, improve
type safety, and refine HTTP handler and SSH git operations.
2026-04-27 08:28:09 +08:00
ZhenYi
02b7a5beda feat(gitserver): add /robots.txt to disallow all crawlers
Some checks are pending
CI / Rust Lint & Check (push) Waiting to run
CI / Rust Tests (push) Waiting to run
CI / Frontend Lint & Type Check (push) Waiting to run
CI / Frontend Build (push) Blocked by required conditions
- Returns Disallow: / for all user-agents
- Points crawlers to main site sitemap via APP_GIT_HTTP_DOMAIN
2026-04-26 00:16:21 +08:00
ZhenYi
10836730ed feat: add health endpoints and Prometheus metrics to git-hook and email-worker
Some checks are pending
CI / Rust Lint & Check (push) Waiting to run
CI / Rust Tests (push) Waiting to run
CI / Frontend Lint & Type Check (push) Waiting to run
CI / Frontend Build (push) Blocked by required conditions
Health monitoring:
- gitserver: /health endpoint on port 8021 (DB + Redis ping)
- git-hook: hyper health server on port 8083 with /health
- email-worker: hyper health server on port 8084 with /health
- K8s probes updated to httpGet for all three services

Metrics (via /metrics endpoint):
- git-hook: hook_tasks_total/success/failed/locked/retried/exhausted,
  hook_sync_branches/tags_changed_total
- email: email_queued/consumed/sent/failed_total,
  email_validation_skipped/build_errors/send_attempts_total
2026-04-25 23:45:48 +08:00
ZhenYi
0c1a9ddf98 refactor(git): migrate libs/git from slog to tracing
- Remove all use slog::* imports and log: slog::Logger fields
- ssh/handle.rs: replace slog macro chains with tracing::{info!, warn!,
  error!, debug!}; remove log field from GitSshHandle
- ssh/authz.rs, ssh/mod.rs, ssh/server.rs: remove slog Logger fields
- http/: auth.rs, handler.rs, mod.rs, routes.rs: remove slog usage
- hook/: pool worker, sync modules, webhook_dispatch.rs: remove slog
2026-04-21 22:29:26 +08:00
ZhenYi
81e6ee3d48 feat(observability): Phase 1-5 slog structured logging across platform
Phase 1: add libs/observability crate (build_logger, instance_id);
  remove duplicate logger init from 4 crates
Phase 2: Actix-web RequestLogger with trace_id; MetricsMiddleware + HttpMetrics
Phase 3: Git SSH handle.rs slog struct; HTTP handler Logger kv
Phase 4: AI client eprintln -> slog warn; billing ai_usage_recorded log
Phase 5: SessionManager slog; workspace alert slog 2.x syntax
2026-04-21 13:44:12 +08:00
ZhenYi
ef61b193c4 fix(git/hook): refine Redis queue worker, remove dead code, fix warnings
- pool/mod.rs: pass shared http_client Arc to HookWorker
- worker.rs: remove double-locking (sync() manages its own lock),
  await all webhook handles before returning, share http_client,
  hoist namespace query out of loop
- redis.rs: atomic NAK via Lua script (LREM + LPUSH in one eval)
- sync/lock.rs: increase LOCK_TTL from 60s to 300s for large repos
- sync/mod.rs: split sync/sync_work, fsck_only/fsck_work, gc_only/gc_work
  so callers can choose locked vs lock-free path; run_gc + sync_skills
  outside the DB transaction
- hook/mod.rs: remove unused http field from HookService
- ssh/mod.rs, http/mod.rs: remove unused HookService/http imports
2026-04-17 13:05:07 +08:00
ZhenYi
8fb2436f22 feat(git): add Redis-backed hook worker with per-repo distributed locking
- pool/worker.rs: single-threaded consumer that BLMPOPs from Redis queues
  sequentially. K8s replicas provide HA — each pod runs one worker.
- pool/redis.rs: RedisConsumer with BLMOVE atomic dequeue, ACK/NAK, and
  retry-with-json support.
- pool/types.rs: HookTask, TaskType, PoolConfig (minimal — no pool metrics).
- sync/lock.rs: Redis SET NX EX per-repo lock to prevent concurrent workers
  from processing the same repo. Lock conflicts are handled by requeueing
  without incrementing retry count.
- hook/mod.rs: HookService.start_worker() spawns the background worker.
- ssh/mod.rs / http/mod.rs: ReceiveSyncService RPUSHes to Redis queue.
  Both run_http and run_ssh call start_worker() to launch the consumer.
- Lock conflicts (GitError::Locked) in the worker are requeued without
  incrementing retry_count so another worker can pick them up.
2026-04-17 12:33:58 +08:00
ZhenYi
eeb99bf628 refactor(git): drop hook pool, sync execution is now direct and sequential
Some checks are pending
CI / Rust Lint & Check (push) Waiting to run
CI / Rust Tests (push) Waiting to run
CI / Frontend Lint & Type Check (push) Waiting to run
CI / Frontend Build (push) Blocked by required conditions
- Remove entire pool/ directory (RedisConsumer, CpuMonitor, LogStream, HookTask, TaskType)
- Remove Redis distributed lock (acquire_lock/release_lock) — K8s StatefulSet
  scheduling guarantees exclusive access per repo shard
- Remove sync/lock.rs, sync/remote.rs, sync/status.rs (dead code)
- Remove hook/event.rs (GitHookEvent was never used)
- New HookService exposes sync_repo / fsck_repo / gc_repo directly
- ReceiveSyncService now calls HookService inline instead of LPUSH to Redis queue
- sync/mod.rs: git2 operations wrapped in spawn_blocking for Send safety
  (git2 types are not Send — async git2 operations must not cross await points)
- scripts/push.js: drop 'frontend' from docker push list (embedded into static binary)
2026-04-17 12:22:09 +08:00
ZhenYi
3de4fff11d feat(service): improve model sync and harden git HTTP/SSH stability
Some checks are pending
CI / Rust Lint & Check (push) Waiting to run
CI / Rust Tests (push) Waiting to run
CI / Frontend Lint & Type Check (push) Waiting to run
CI / Frontend Build (push) Blocked by required conditions
Model sync:
- Filter OpenRouter models by what the user's AI client can actually access,
  before upserting metadata (avoids bloating with inaccessible models).
- Fall back to direct endpoint sync when no OpenRouter metadata matches
  (handles Bailian/MiniMax and other non-OpenRouter providers).

Git stability fixes:
- SSH: add 5s timeout on stdin flush/shutdown in channel_eof and
  cleanup_channel to prevent blocking the event loop on unresponsive git.
- SSH: remove dbg!() calls from production code paths.
- HTTP auth: pass proper Logger to SshAuthService instead of discarding
  all auth events to slog::Discard.

Dependencies:
- reqwest: add native-tls feature for HTTPS on Windows/Linux/macOS.
2026-04-17 00:13:40 +08:00
ZhenYi
f5ab554d6b fix(git): add LFS upload size limits and fix HTTP rate limiter read/write counter
- Add LFS_MAX_OBJECT_SIZE (50 GiB) and validate object sizes in both the
  batch advisory check and the upload_object streaming loop to prevent
  unbounded disk usage from malicious clients
- Fix HTTP rate limiter: track read_count and write_count separately so
  a burst of writes cannot exhaust the read budget (previously all
  operations incremented read_count regardless of type)
2026-04-16 20:14:13 +08:00
ZhenYi
cef4ff1289 fix(git): harden HTTP and SSH git transports for robustness
HTTP:
- Return Err(...) instead of Ok(HttpResponse::...) for error cases so
  actix returns correct HTTP status codes instead of 200
- Add 30s timeout on info_refs and handle_git_rpc git subprocess calls
- Add 1MB pre-PACK limit to prevent memory exhaustion on receive-pack
- Enforce branch protection rules (forbid push/force-push/deletion/tag)
- Simplify graceful shutdown (remove manual signal handling)

SSH:
- Fix build_git_command: use block match arms so chained .arg() calls
  are on the Command, not the match expression's () result
- Add MAX_RETRIES=5 to forward() data-pump loop to prevent infinite
  spin on persistent network failures
- Fall back to raw path if canonicalize() fails instead of panicking
- Add platform-specific git config paths (/dev/null on unix, NUL on win)
- Start rate limiter cleanup background task so HashMap doesn't grow
  unbounded over time
- Derive Clone on RateLimiter so SshRateLimiter::start_cleanup works
2026-04-16 20:11:18 +08:00
ZhenYi
93cfff9738 init 2026-04-15 09:08:09 +08:00