Compare commits

...

61 Commits

Author SHA1 Message Date
zhenyi
1e82d22048 chore(git): ignore .codegraph directory in git tracking
- Added .codegraph to .gitignore to prevent tracking build artifacts
- Maintains clean repository state by excluding generated files
2026-06-01 22:05:41 +08:00
zhenyi
c7c490bf77 feat: add env example, chart config, and track lib 2026-06-01 22:05:03 +08:00
zhenyi
9bc0e742bc refactor: update frontend components and pages 2026-06-01 22:04:53 +08:00
zhenyi
10baa7fbd2 refactor: update git, email, AI libs and app entries 2026-06-01 22:04:46 +08:00
zhenyi
ff5beeca31 refactor: update API and service layers 2026-06-01 22:04:38 +08:00
zhenyi
079ea3a5cf refactor: update channel and model layers 2026-06-01 22:04:31 +08:00
zhenyi
734e1c4cc8 refactor: update infrastructure libs (config, db, cache, queue, storage, migrate) 2026-06-01 22:04:25 +08:00
zhenyi
e44f3d13c4 chore: update docker configuration 2026-06-01 22:04:17 +08:00
zhenyi
b65ea19b85 chore: update config and build files 2026-06-01 22:04:09 +08:00
zhenyi
e771db0c70 feat(channel): add X embed card and link parser 2026-05-31 13:12:41 +08:00
zhenyi
980cd54b66 feat(repo): add repository components 2026-05-31 13:12:36 +08:00
zhenyi
1ef37786b1 feat(landing): add landing page components 2026-05-31 13:12:32 +08:00
zhenyi
8b8c9e5e33 feat(api): add rest embed channel handler 2026-05-31 13:12:28 +08:00
zhenyi
fcddc06cfe fix(workplan): update workplan chat and settings components 2026-05-31 13:12:14 +08:00
zhenyi
7a1b03060e fix(repo): update repository page components 2026-05-31 13:12:08 +08:00
zhenyi
71d2e059a6 fix(issues): update issue page components 2026-05-31 13:12:01 +08:00
zhenyi
e1cbbc1e0f fix(workspace): update workspace, me, and settings pages 2026-05-31 13:11:56 +08:00
zhenyi
879aafa3fa fix(socket): add missing article events and notification types 2026-05-31 13:11:51 +08:00
zhenyi
29e6f6214d fix(channel): update channel components and article draft 2026-05-31 13:11:44 +08:00
zhenyi
82475e95d5 fix(auth): update auth page components 2026-05-31 13:11:37 +08:00
zhenyi
71f90bcd4d fix(ai-elements): update AI element and mention-chip components 2026-05-31 13:11:32 +08:00
zhenyi
0f7b05f7ef fix(components): update shell components and settings modal null check 2026-05-31 13:11:26 +08:00
zhenyi
a96222cc28 fix(ui): add asChild support to Button and update theme styles 2026-05-31 13:11:19 +08:00
zhenyi
6a0fcf5343 chore: update dependencies and configuration 2026-05-31 13:11:13 +08:00
zhenyi
779e4eae2f feat(channel): add article feed and composer with room type support
- Add ArticleFeed component for article-based channels
- Implement ArticleComposer with draft persistence
- Add Newspaper icon for article room type
- Update ChannelPage to conditionally render article feed vs message view
- Add article-related API endpoints and models
- Reset thread view when switching rooms
- Add room type check in channel sidebar
- Update CSS to hide scrollbars globally
- Add gRPC message size limit configuration
- Fix git diff tree handling
2026-05-31 03:09:49 +08:00
zhenyi
f947c931cd actor(visibilityref): update function visibility and formatting across modules 2026-05-30 22:54:09 +08:00
zhenyi
9ffc7c9fb3 refactor(git): update git command to use trusted parameters
- Modified commit creation to use trusted GitCommandParams
- Ensured secure command execution with trusted parameter validation
- Maintained existing commit OID handling and update reference logic
2026-05-30 20:06:44 +08:00
zhenyi
4f76816de8 feat: add mention-textarea-overlay and repo-mention-popover components 2026-05-30 15:08:27 +08:00
zhenyi
dbcdf04817 fix: update socket fallback 2026-05-30 15:08:22 +08:00
zhenyi
3b63a43404 feat: update workplan chat components (conversation, sidebar, bubble, utils) 2026-05-30 15:08:18 +08:00
zhenyi
600ed2de35 feat: update workspace settings general page 2026-05-30 15:08:14 +08:00
zhenyi
c7d67960b7 feat: update workspace repo pages (branches, code, commits, layout, settings, tags) 2026-05-30 15:08:10 +08:00
zhenyi
a771dcf5be feat: update workspace issues page 2026-05-30 15:08:05 +08:00
zhenyi
b6a4bd0210 feat: update workspace channel components (header, sidebar, thread-panel, composer, etc) 2026-05-30 15:08:01 +08:00
zhenyi
04798b5adb feat: update appearance settings page 2026-05-30 15:07:56 +08:00
zhenyi
b489296b08 feat: update personal chat conversation page 2026-05-30 15:07:51 +08:00
zhenyi
f658c5ae96 feat: update UI components (badge, button, card, input, textarea) 2026-05-30 15:07:46 +08:00
zhenyi
9b1f764ab8 feat: update shell components (rail, settings-sidebar, workspace-sidebar) 2026-05-30 15:07:42 +08:00
zhenyi
db5b54025b feat: update App entry and global styles 2026-05-30 15:07:36 +08:00
zhenyi
79b03a90a7 refactor: update Socket.IO actix integration 2026-05-30 15:07:31 +08:00
zhenyi
3e07fedd0c refactor: update agent service (context, run, sse) 2026-05-30 15:07:26 +08:00
zhenyi
4d2e4d8b36 refactor: update channel pagination and reconnect 2026-05-30 15:07:20 +08:00
zhenyi
22192301a0 refactor: update channel HTTP layer (out_event, types, ws) 2026-05-30 15:07:16 +08:00
zhenyi
2b7777adbc refactor: update channel HTTP handlers (ban, message, room, thread) 2026-05-30 15:07:12 +08:00
zhenyi
25b73e1054 refactor: update channel bus and thread event 2026-05-30 15:07:07 +08:00
zhenyi
d0155c66d2 refactor: update Rust API channel layer (interact, message, mod) 2026-05-30 15:07:03 +08:00
zhenyi
70818a7c71 chore: update Cargo.toml dependencies 2026-05-30 15:06:58 +08:00
zhenyi
c3370707af fix: remove unused imports and parameter in thread-sidebar 2026-05-30 15:00:06 +08:00
zhenyi
41bd76c45b fix: remove unused imports and variables in thread-pane 2026-05-30 15:00:01 +08:00
zhenyi
ccc344debd fix: make onSend callback async to match Promise<void> return type 2026-05-30 14:59:57 +08:00
zhenyi
3d54df27b4 fix: remove dead code and fix isDark property reference on ThemeColors 2026-05-30 14:59:53 +08:00
zhenyi
9ec6bce9c9 fix: remove unused AnimatePresence import and sessions variable 2026-05-30 14:59:49 +08:00
zhenyi
1b5e9799eb fix: add missing import and export for ApproveWorkspaceJoinApply 2026-05-30 14:59:45 +08:00
zhenyi
a835610737 feat: 1.0 2026-05-30 01:38:40 +08:00
zhenyi
e1330451a5 feat: 1.0 2026-05-30 01:38:27 +08:00
zhenyi
2b543f5e37 refactor(ui): remove deprecated project components
Delete unused project components:
- Board: BoardColumn, BoardHeader, BoardListView, BoardModals, KanbanBoard
- Channel: RoomSettingsModal, AiSettings
- Issue: IssueSidebar, ReactionBar
- Skills: CreateSkillDialog, DeleteSkillDialog
- ProjectCreateMenuModal
2026-05-20 13:39:18 +08:00
zhenyi
f6f69a063e refactor(ui): remove deprecated me page components
Delete unused components:
- ActivityTimeline, ContributionHeatmap, CreateProjectModal
- FollowerCardList, MeSidebar, NotificationList
- ProfileHeader, ProjectList, RepoList, UserCardList
- LoadingSpinner (replaced by LoadingState)
2026-05-20 13:38:36 +08:00
zhenyi
5827d561db refactor(agent): remove deprecated chat modules
Delete deprecated chat components:
- agent_profile, context, message_builder
- nonstreaming_execution, session_recording

These are superseded by the consolidated chat module.
2026-05-20 13:38:25 +08:00
zhenyi
d3de12717d refactor(room): consolidate AI service modules
Delete redundant AI service implementations:
- ai_mode_streaming variants
- ai_react_streaming variants
- process_ai

These are superseded by the consolidated ai_service module.
2026-05-20 13:38:11 +08:00
zhenyi
6dbbd22036 chore: remove workspace module
Delete deprecated workspace module:
- libs/api/workspace: workspace API handlers
- libs/service/workspace: workspace business logic
2026-05-20 13:37:43 +08:00
zhenyi
d59647d9a8 chore: remove gingress ingress controller and proxy
Delete unused ingress controller and proxy components:
- apps/gingress: Kubernetes ingress controller
- libs/gingress-proxy: Gateway proxy with rate limiting, TLS, etc.
- docker/gingress.Dockerfile
2026-05-20 13:37:32 +08:00
3770 changed files with 146919 additions and 244790 deletions

4
.clippy.toml Normal file
View File

@ -0,0 +1,4 @@
# Clippy configuration
doc-valid-idents = ["GitHub", "GitLab", "TypeScript", "WebSocket", "PostgreSQL", "Redis", "OpenAI"]
avoid-breaking-exported-api = true
disallowed-types = []

View File

@ -1,13 +1,76 @@
.git/
.idea/
.vscode/
node_modules/
*.log
.env
.env.local
.env.*.local
# Git
.git
.gitignore
# Exclude all target/ content, then selectively re-include release binaries
# IDE
.idea
.vscode
*.swp
*.swo
*~
# Rust build artifacts
target/
!target/release/
!target/x86_64-unknown-linux-gnu/release/
**/target/
# Node.js
node_modules/
npm-debug.log*
yarn-debug.log*
yarn-error.log*
pnpm-debug.log*
bun.lockb
# Build output
dist/
build/
# Environment and secrets
.env
.env.*
!.env.example
# Docker
docker-compose*.yml
.dockerignore
# Documentation
*.md
LICENSE
doc/
# Test and CI
tests/
__tests__/
*.test.*
*.spec.*
.github/
.gitlab/
.circleci/
# OS files
.DS_Store
Thumbs.db
desktop.ini
# Logs
logs/
*.log
# Temporary files
tmp/
temp/
.tmp/
# Certificates (use secrets in production)
cert/
# Data directories
data/
# Agent configs
.agent/
.agents/
.claude/
CLAUDE.md
AGENTS.md

42
.editorconfig Normal file
View File

@ -0,0 +1,42 @@
root = true
[*]
charset = utf-8
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
[*.{js,ts,jsx,tsx,json,jsonc,md,yaml,yml,toml}]
indent_style = space
indent_size = 2
[*.py]
indent_style = space
indent_size = 4
[*.go]
indent_style = tab
indent_size = unset
tab_width = 8
[*.rs]
indent_style = space
indent_size = 4
[Makefile]
indent_style = tab
indent_size = unset
[Dockerfile]
indent_style = space
indent_size = 2
[*.md]
trim_trailing_whitespace = false
[*.{yml,yaml}]
indent_style = space
indent_size = 2
[*.toml]
indent_style = space
indent_size = 2

View File

@ -1,136 +1,65 @@
# =============================================================================
# Required - 程序启动必须配置
# =============================================================================
# ============================================================
# GitDataAI Development Environment Variables
# Copy to .env and adjust as needed
# ============================================================
# 数据库连接
APP_DATABASE_URL=postgresql://user:password@localhost:5432/dbname
APP_DATABASE_SCHEMA_SEARCH_PATH=public
# ── Database ───────────────────────────────────────────────
POSTGRES_USER=gitdata
POSTGRES_PASSWORD=gitdata123
POSTGRES_DB=app
# Redis支持多节点逗号分隔
APP_REDIS_URL=redis://localhost:6379
# APP_REDIS_URLS=redis://localhost:6379,redis://localhost:6378
# ── MinIO (S3-compatible storage) ──────────────────────────
MINIO_ROOT_USER=admin
MINIO_ROOT_PASSWORD=mysecret123
# AI 服务
APP_AI_BASIC_URL=https://api.openai.com/v1
APP_AI_API_KEY=sk-xxxxx
# ── Application ────────────────────────────────────────────
APP_API_PORT=8080
APP_NAME=gitdata
APP_DOMAIN_URL=http://localhost
APP_SESSION_SECRET=supersecretdevkey123
# Embedding + 向量检索
APP_EMBED_MODEL_BASE_URL=https://api.openai.com/v1
APP_EMBED_MODEL_API_KEY=sk-xxxxx
APP_EMBED_MODEL_NAME=text-embedding-3-small
APP_EMBED_MODEL_DIMENSIONS=1536
# ── Database Connection ────────────────────────────────────
APP_DATABASE_URL=postgres://gitdata:gitdata123@localhost:5432/app
# ── Redis ──────────────────────────────────────────────────
APP_REDIS_URLS=redis://localhost:6379
# ── Qdrant Vector DB ───────────────────────────────────────
APP_QDRANT_URL=http://localhost:6333
# APP_QDRANT_API_KEY=
# SMTP 邮件
APP_SMTP_HOST=smtp.example.com
APP_SMTP_PORT=587
APP_SMTP_USERNAME=noreply@example.com
APP_SMTP_PASSWORD=xxxxx
APP_SMTP_FROM=noreply@example.com
APP_SMTP_TLS=true
APP_SMTP_TIMEOUT=30
# ── NATS ───────────────────────────────────────────────────
NATS_URL=nats://localhost:4222
# 文件存储
APP_AVATAR_PATH=/data/avatars
# Git 仓库存储根目录
APP_REPOS_ROOT=/data/repos
# ── Storage (S3) ───────────────────────────────────────────
APP_STORAGE_BACKEND=s3
APP_STORAGE_S3_BUCKET=gitdata
APP_STORAGE_S3_REGION=us-east-1
APP_STORAGE_S3_ENDPOINT_URL=http://localhost:9000
APP_STORAGE_S3_ACCESS_KEY_ID=admin
APP_STORAGE_S3_SECRET_ACCESS_KEY=mysecret123
APP_STORAGE_S3_FORCE_PATH_STYLE=true
# =============================================================================
# Domain / URL可选有默认值
# =============================================================================
# ── Git Service (gitpod) ───────────────────────────────────
APP_GIT_RPC_ADDR=0.0.0.0
APP_GIT_RPC_PORT=5030
APP_GIT_HTTP_PORT=5023
APP_SSH_PORT=5022
APP_DOMAIN_URL=http://127.0.0.1
# APP_STATIC_DOMAIN=
# APP_MEDIA_DOMAIN=
# APP_GIT_HTTP_DOMAIN=
# ── GitSync Health ─────────────────────────────────────────
APP_GITSYNC_HEALTH_PORT=5083
# =============================================================================
# Database Pool可选有默认值
# =============================================================================
# ── SMTP (Email) ───────────────────────────────────────────
# For development, use MailHog: docker run -d -p 1025:1025 -p 8025:8025 mailhog/mailhog
APP_SMTP_HOST=localhost
APP_SMTP_PORT=1025
APP_SMTP_USERNAME=dev
APP_SMTP_PASSWORD=dev
APP_SMTP_FROM=Gitdata <noreply@localhost>
# APP_DATABASE_MAX_CONNECTIONS=10
# APP_DATABASE_MIN_CONNECTIONS=2
# APP_DATABASE_IDLE_TIMEOUT=60000 (milliseconds, default: 60s)
# APP_DATABASE_MAX_LIFETIME=300000 (milliseconds, default: 300s)
# APP_DATABASE_CONNECTION_TIMEOUT=5000 (milliseconds, default: 5s)
# APP_DATABASE_REPLICAS=
# APP_DATABASE_HEALTH_CHECK_INTERVAL=30
# APP_DATABASE_RETRY_ATTEMPTS=3
# APP_DATABASE_RETRY_DELAY=5
# =============================================================================
# Redis Pool可选有默认值
# =============================================================================
# APP_REDIS_POOL_SIZE=10
# APP_REDIS_CONNECT_TIMEOUT=5
# APP_REDIS_ACQUIRE_TIMEOUT=5
# =============================================================================
# SSH可选有默认值
# =============================================================================
# APP_SSH_DOMAIN=
# APP_SSH_PORT=22
# APP_SSH_SERVER_PRIVATE_KEY=
# APP_SSH_SERVER_PUBLIC_KEY=
# =============================================================================
# Logging可选有默认值
# =============================================================================
# APP_LOG_LEVEL=info
# APP_LOG_FORMAT=json
# APP_LOG_FILE_ENABLED=false
# APP_LOG_FILE_PATH=./logs
# APP_LOG_FILE_ROTATION=daily
# APP_LOG_FILE_MAX_FILES=7
# APP_LOG_FILE_MAX_SIZE=104857600
# OpenTelemetry可选默认关闭
# APP_OTEL_ENABLED=false
# APP_OTEL_ENDPOINT=http://localhost:5080/api/default/v1/traces
# APP_OTEL_SERVICE_NAME=
# APP_OTEL_SERVICE_VERSION=
# APP_OTEL_AUTHORIZATION=
# APP_OTEL_ORGANIZATION=
# =============================================================================
# NATS / Hook Pool可选有默认值
# =============================================================================
# HOOK_POOL_MAX_CONCURRENT=CPU 核数)
# HOOK_POOL_CPU_THRESHOLD=80.0
# HOOK_POOL_REDIS_LIST_PREFIX={hook}
# HOOK_POOL_REDIS_LOG_CHANNEL=hook:logs
# HOOK_POOL_REDIS_BLOCK_TIMEOUT=5
# HOOK_POOL_REDIS_MAX_RETRIES=3
# HOOK_POOL_WORKER_ID=(随机 UUID
# =============================================================================
# Frontend (Vite) — 前端运行环境变量
# =============================================================================
# API 基础 URL为空时使用 Vite dev 代理 /api -> localhost:8080
# VITE_API_BASE_URL=http://localhost:8080
# 前端 WebSocket 连接地址(开发模式通过 Vite 代理)
VITE_WS_URL=ws://localhost:5080
# API URL前端 API 调用,通过 Vite 代理时可为空)
VITE_API_URL=
# WebSocket 连接模式: "raw-ws" | "socketio"
VITE_WS_MODE=raw-ws
# =============================================================================
# Frontend: Grafana Faro (RUM) — 前端性能监控(可选)
# =============================================================================
# VITE_FARO_ENABLED=false
# VITE_FARO_URL=https://faro.example.com/collect
# VITE_FARO_API_KEY=
# VITE_FARO_APP_NAME=GitDataAIWeb
# VITE_FARO_APP_ENV=production
# VITE_FARO_APP_VERSION=0.0.1
# ── AI / Embed ─────────────────────────────────────────────
APP_AI_BASIC_URL=http://localhost:11434/v1
APP_AI_API_KEY=ollama
APP_EMBED_MODEL_BASE_URL=http://localhost:11434/v1
APP_EMBED_MODEL_API_KEY=ollama
APP_EMBED_MODEL_NAME=nomic-embed-text
APP_EMBED_MODEL_DIMENSIONS=768

85
.gitignore vendored
View File

@ -1,29 +1,64 @@
# Rust build artifacts
/target
node_modules
.claude
.zed
.vscode
.idea
**/target/
# Rust IDE and tooling
.idea/
.vscode/
*.swp
*.swo
*~
# Environment files
.env
.env.local
dist
deploy/secrets.yaml
.codex
.qwen
.opencode
.omc
AGENT.md
ARCHITECTURE.md
.agents
.agents.md
.next
.env.*.local
.env.production
# OS files
.DS_Store
Thumbs.db
desktop.ini
# Node.js
node_modules/
coverage/
.pnpm-store/
pnpm-lock.yaml
package-lock.json
yarn.lock
.gemini
.omg
/.sqry
deploy/.server.yaml
npm-debug.log*
yarn-debug.log*
yarn-error.log*
pnpm-debug.log*
# Build output
dist/
build/
.codegraph
# Logs
logs/
*.log
# Data and certificates
data/
cert/
# Docker
docker-compose.override.yml
.docker/
# Agent configs
.claude/
.codex/
.agent/
.agents/
CLAUDE.md
AGENTS.md
migrate.sh
# Temporary files
tmp/
temp/
.tmp/
# Backup files
*.bak
*.backup
*~
chart/ConfigMap.yaml
chart/pvc

View File

@ -1,11 +0,0 @@
{
"mcpServers": {
"shadcn": {
"command": "npx",
"args": [
"shadcn@latest",
"mcp"
]
}
}
}

View File

@ -1,7 +0,0 @@
node_modules/
coverage/
.pnpm-store/
pnpm-lock.yaml
package-lock.json
pnpm-lock.yaml
yarn.lock

View File

@ -1,11 +0,0 @@
{
"endOfLine": "lf",
"semi": false,
"singleQuote": false,
"tabWidth": 2,
"trailingComma": "es5",
"printWidth": 80,
"plugins": ["prettier-plugin-tailwindcss"],
"tailwindStylesheet": "src/index.css",
"tailwindFunctions": ["cn", "cva"]
}

7087
Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@ -1,192 +1,5 @@
[workspace]
members = [
# "libs/frontend",
"libs/models",
"libs/session",
"libs/git",
"libs/email",
"libs/queue",
"libs/room",
"libs/config",
"libs/service",
"libs/db",
"libs/api",
"libs/transport",
"libs/observability",
"libs/avatar",
"libs/agent",
"libs/migrate",
"libs/fctool",
"libs/gingress-proxy",
"apps/migrate",
"apps/app",
"apps/git-hook",
"apps/gitserver",
"apps/email",
"apps/static",
"apps/metrics",
"apps/gingress",
]
resolver = "3"
[workspace.dependencies]
models = { path = "libs/models" }
session = { path = "libs/session" }
git = { path = "libs/git" }
email = { path = "libs/email" }
queue = { path = "libs/queue" }
room = { path = "libs/room" }
config = { path = "libs/config" }
service = { path = "libs/service" }
db = { path = "libs/db" }
api = { path = "libs/api" }
agent = { path = "libs/agent" }
observability = { path = "libs/observability" }
avatar = { path = "libs/avatar" }
migrate = { path = "libs/migrate" }
fctool = { path = "libs/fctool" }
transport = { path = "libs/transport" }
metrics-aggregator = { path = "apps/metrics" }
gingress-proxy = { path = "libs/gingress-proxy" }
gingress = { path = "apps/gingress" }
sea-query = "1.0.0-rc.33"
actix-web = "4.13.0"
actix-files = "0.6.10"
actix-cors = "0.7.1"
actix-session = "0.11.0"
actix-ws = "0.4.0"
actix-multipart = "0.7.2"
actix-analytics = "1.2.1"
actix-jwt-session = "1.0.7"
actix-csrf = "0.8.0"
metrics = "0.24.5"
actix-rt = "2.11.0"
actix = "0.13"
async-stream = "0.3"
actix-service = "2.0.3"
actix-utils = "3.0.1"
redis = "1.1.0"
anyhow = "1.0.102"
derive_more = "2.1.1"
blake3 = "1.8.3"
argon2 = "0.5.3"
thiserror = "2.0.18"
password-hash = "0.6.0"
awc = "3.8.2"
bstr = "1.12.1"
captcha-rs = "0.5.0"
deadpool-redis = "0.23.0"
deadpool = "0.13.0"
dotenv = "0.15.0"
env_logger = "0.11.10"
brotli = "7.0"
flate2 = "1.1.9"
git2 = "0.20.4"
slog = "2.8.2"
git2-ext = "1.0.0"
git2-hooks = "0.7.0"
futures = "0.3.32"
futures-util = "0.3.32"
globset = "0.4.18"
hex = "0.4.3"
lettre = { version = "0.11.19", default-features = false, features = ["tokio1-rustls-tls", "smtp-transport", "builder", "pool"] }
mime = "0.3.17"
mime_guess2 = "2.3.1"
opentelemetry = "0.31.0"
opentelemetry-otlp = { version = "0.31.0", features = ["http-proto", "trace"] }
opentelemetry_sdk = { version = "0.31.0", features = ["rt-tokio"] }
opentelemetry-http = "0.31.0"
prost = "0.14.3"
prost-build = "0.14.3"
qdrant-client = "1.17.0"
prost-types = "0.14.3"
rand = "0.10.0"
russh = { version = "0.60.2", default-features = false, features = ["ring", "rsa"] }
hmac = { version = "0.13" }
hkdf = "0.13.0"
sha1_smol = "1.0.1"
rsa = { version = "0.9.7", package = "rsa" }
reqwest = { version = "0.13.2", default-features = false }
dotenvy = "0.15.7"
# aws-lc-sys requires NASM on Windows, so we use local filesystem storage instead of S3
# aws-sdk-s3 = "1.127.0"
sea-orm = "2.0.0-rc.37"
sea-orm-migration = "2.0.0-rc.37"
sha1 = "0.11"
sha2 = "0.11"
sysinfo = "0.39.1"
ssh-key = "0.7.0-rc.9"
tar = "0.4.45"
zip = "8.3.1"
tokenizer = "0.1.2"
tiktoken-rs = "0.11.0"
regex = "1.12.3"
jsonwebtoken = "10.3.0"
once_cell = "1.21.4"
async-trait = "0.1.89"
fs2 = "0.4.3"
image = "0.25.10"
tokio = "1.50.0"
tokio-util = "0.7.18"
tokio-stream = { version = "0.1.18", features = ["sync"] }
url = "2.5.8"
tower = "0.5"
num_cpus = "1.17.0"
ring = "0.17"
rustls = { version = "0.23", default-features = false, features = ["ring", "std", "tls12"] }
clap = "4.6.0"
time = "0.3.47"
chrono = "0.4.44"
tracing = "0.1.44"
tracing-subscriber = { version = "0.3.23", features = ["env-filter", "json", "tracing-log"] }
tracing-opentelemetry = "0.32.1"
tonic = "0.14.5"
tonic-build = "0.14.5"
uuid = "1.22.0"
hostname = "0.4"
utoipa = { version = "5.4.0", features = ["chrono", "uuid"] }
rust_decimal = "1.40.0"
walkdir = "2.5.0"
calamine = "0.26"
csv = "1.3"
lopdf = "0.34"
pulldown-cmark = "0.12"
quick-xml = { version = "0.37", features = ["serialize"] }
sqlparser = "0.55"
lazy_static = "1.5"
chacha20poly1305 = "0.10"
md5 = "0.7"
moka = "0.12.15"
dashmap = "7.0.0-rc2"
serde = "1.0.228"
serde_json = "1.0.149"
serde_yaml = "0.9.33"
serde_bytes = "0.11.19"
phf = "0.13.1"
phf_codegen = "0.13.1"
base64 = "0.22.1"
base64ct = "1"
p256 = { version = "0.13", features = ["ecdsa", "std"] }
# http version varies per-crate (pingora needs 1.x, actix needs 0.2)
hyper = "0.14"
tempfile = "3"
rig-core = { version = "0.36.0", default-features = false }
tokio-tungstenite = { version = "0.29.0", features = [] }
async-nats = { version = "0.48.0", features = [] }
kube = { version = "3.1.0", features = ["runtime", "derive"] }
k8s-openapi = { version = "0.27", features = ["v1_31"] }
pingora = { version = "0.8", features = ["proxy"] }
pingora-proxy = "0.8"
pingora-load-balancing = "0.8"
pingora-cache = "0.8"
rustls-pemfile = "2"
[workspace.package]
version = "0.2.9"
version = "1.0.0"
edition = "2024"
authors = []
description = ""
@ -198,6 +11,33 @@ keywords = []
categories = []
documentation = ""
[workspace]
members = [
"app/email",
"app/gitdata",
"app/gitpod",
"app/gitsync",
"lib/ai",
"lib/api",
"lib/cache",
"lib/channel",
"lib/config",
"lib/db",
"lib/email",
"lib/git",
"lib/issues",
"lib/migrate",
"lib/model",
"lib/queue",
"lib/service",
"lib/session",
"lib/storage",
"lib/parsefile",
"lib/socketio",
"lib/track"
]
resolver = "3"
[workspace.lints.rust]
unsafe_code = "warn"
@ -205,36 +45,107 @@ unsafe_code = "warn"
unwrap_used = "warn"
expect_used = "warn"
[profile.dev]
debug = 1
incremental = true
codegen-units = 256
[workspace.dependencies]
ai = { path = "lib/ai" }
api = { path = "lib/api" }
cache = { path = "lib/cache" }
channel = { path = "lib/channel" }
config = { path = "lib/config" }
db = { path = "lib/db" }
email = { path = "lib/email" }
git = { path = "lib/git" }
issues = { path = "lib/issues" }
migrate = { path = "lib/migrate" }
model = { path = "lib/model" }
queue = { path = "lib/queue" }
service = { path = "lib/service" }
session = { path = "lib/session" }
storage = { path = "lib/storage" }
parsefile = { path = "lib/parsefile"}
socketio = { path = "lib/socketio" }
track = { path = "lib/track" }
[profile.release]
lto = "thin"
codegen-units = 1
strip = true
opt-level = 3
leptos = "0.8.19"
leptos_actix = "0.8.7"
leptos_meta = "0.8.6"
leptos_router = "0.8.13"
server_fn = { version = "0.8.10", features = ["actix"] }
actix-http = "3.11"
actix-ws = "0.4.0"
urlencoding = "2.1"
serde_urlencoded = "0.7"
[profile.dev.package.num-bigint-dig]
opt-level = 3
[package]
name = "workspace"
version.workspace = true
edition.workspace = true
authors.workspace = true
description.workspace = true
repository.workspace = true
readme.workspace = true
homepage.workspace = true
license.workspace = true
keywords.workspace = true
categories.workspace = true
documentation.workspace = true
[lib]
path = "lib.rs"
crate-type = ["lib"]
juniper = "0.17.1"
ractor = "0.15.13"
ractor_cluster = "0.15.13"
async-nats = "0.48.0"
petgraph = "0.8.3"
async-openai = "0.40.0"
rig-core = { version = "0.36.0", default-features = false, features = ["derive"] }
schemars = "1.2.1"
tokio-stream = "0.1.18"
duct = "1.1.1"
lettre = "0.11.22"
actix-web = "4"
jsonwebtoken = { version = "10.4.0", features = ["rust_crypto"] }
futures-util = "0.3"
futures = "0.3.32"
moka = "0.12.15"
tokio = "1.52.3"
redis = "1.2.1"
serde_json = "1.0.149"
indexmap = "2.14.0"
sea-orm-migration = "2.0.0-rc.38"
sea-orm = { version = "2.0.0-rc.38", features = ["sqlx-all","runtime-tokio","rust_decimal","uuid","chrono"]}
async-trait = "0.1.89"
aws-config = "1.8.16"
aws-sdk-s3 = "1.132.0"
rust_decimal = "1.42.0"
utoipa = "5.5.0"
dotenvy = "0.15.7"
anyhow = "1.0.102"
derive_more = "2.1.1"
serde = "1.0.228"
serde_yaml = "0.9.33"
comrak = "0.38"
sqlparser = "0.62.0"
qdrant-client = "1.18.0"
tiktoken-rs = "0.11.0"
tracing-subscriber = "0.3.23"
tracing-actix-web = "0.7"
thiserror = "2.0.18"
uuid = "1.23.1"
git2 = "0.21.0"
gix = { version = "0.83.0", features = ["max-performance-safe", "serde", "merge", "blame", "revision", "blob-diff", "worktree-stream", "worktree-archive", "mailmap"] }
gix-archive = "0.32.0"
gix-worktree-stream = "0.32.0"
num_cpus = "1.17.0"
tracing = "0.1.44"
actix-service = "2.0.3"
actix-rt = "2.11.0"
actix-utils = "3.0.1"
toasty = "0.6.1"
chrono = "0.4.44"
argon2 = "0.5.3"
rand = "0.10.1"
rand_core = { version = "0.10.1", features = ["getrandom"] }
totp-rs = "5.7.1"
url = "2.5.7"
sha2 = "0.11.0"
base64 = "0.22"
tonic = "0.14.6"
tonic-build = "0.14.6"
prost = "0.14.3"
tonic-prost = "0.14.6"
dashmap = "6"
sqlx = "0.9.0"
russh = { version = "0.61.1", features = ["legacy-ed25519-pkcs8-parser"] }
hex = "0.4"
async-stream = "0.3"
tokio-util = "0.7"
password-hash = "0.6.1"
deadpool-redis = { version = "0.23", features = ["cluster"] }
reqwest = { version = "0.13", features = ["json", "rustls", "system-proxy"] }
hmac = "0.13"
mcpkit = "0.5"
miette = "7"

View File

@ -1,21 +0,0 @@
# React + TypeScript + Vite + shadcn/ui
This is a template for a new Vite project with React, TypeScript, and shadcn/ui.
## Adding components
To add components to your app, run the following command:
```bash
npx shadcn@latest add button
```
This will place the ui components in the `src/components` directory.
## Using components
To use the components in your app, import them as follows:
```tsx
import { Button } from "@/components/ui/button"
```

31
app/email/Cargo.toml Normal file
View File

@ -0,0 +1,31 @@
[package]
name = "app-email"
version.workspace = true
edition.workspace = true
authors.workspace = true
description.workspace = true
repository.workspace = true
readme.workspace = true
homepage.workspace = true
license.workspace = true
keywords.workspace = true
categories.workspace = true
documentation.workspace = true
[[bin]]
name = "email-service"
path = "src/main.rs"
[dependencies]
anyhow = { workspace = true }
config = { workspace = true }
email = { workspace = true }
actix-web = { workspace = true }
tracing-actix-web = { workspace = true }
serde_json = { workspace = true }
tokio = { workspace = true, features = ["rt-multi-thread", "macros", "signal"] }
track = { workspace = true }
tracing = { workspace = true }
[lints]
workspace = true

26
app/email/src/context.rs Normal file
View File

@ -0,0 +1,26 @@
use config::AppConfig;
pub struct AppContext {
pub config: AppConfig,
pub metrics: track::MetricsRegistry,
_otel_guard: Option<track::LgtmGuard>,
}
impl AppContext {
pub fn init() -> anyhow::Result<Self> {
let config = AppConfig::load();
let otel_guard = init_tracing(&config)?;
let metrics = track::MetricsRegistry::new();
Ok(Self {
config,
metrics,
_otel_guard: otel_guard,
})
}
}
fn init_tracing(
config: &AppConfig,
) -> anyhow::Result<Option<track::LgtmGuard>> {
track::init_lgtm(config)
}

74
app/email/src/health.rs Normal file
View File

@ -0,0 +1,74 @@
use std::time::Instant;
use actix_web::dev::Service;
use actix_web::{App, HttpResponse, HttpServer, dev::Server, web};
use tracing_actix_web::TracingLogger;
use serde_json;
async fn health() -> HttpResponse {
HttpResponse::Ok().json(serde_json::json!({ "status": "ok" }))
}
async fn metrics(metrics: web::Data<track::MetricsRegistry>) -> HttpResponse {
match metrics.encode() {
Ok(body) => HttpResponse::Ok()
.content_type("text/plain; version=0.0.4")
.body(body),
Err(e) => HttpResponse::InternalServerError()
.content_type("text/plain")
.body(format!("metrics encoding error: {e}")),
}
}
pub fn start_health(
port: u16,
metrics_registry: track::MetricsRegistry,
) -> anyhow::Result<Server> {
tracing::info!("email health endpoint starting on 0.0.0.0:{}", port);
let srv = HttpServer::new(move || {
App::new()
.app_data(web::Data::new(metrics_registry.clone()))
.wrap(TracingLogger::default())
.wrap_fn(|req, srv| {
let method = req.method().clone();
let path = req.path().to_owned();
let peer_addr =
req.connection_info().peer_addr().map(str::to_owned);
let started_at = Instant::now();
let fut = srv.call(req);
async move {
match fut.await {
Ok(res) => {
tracing::info!(
method = %method,
path = %path,
status = res.status().as_u16(),
elapsed_ms = started_at.elapsed().as_millis(),
peer_addr = peer_addr.as_deref().unwrap_or("-"),
"http request"
);
Ok(res)
}
Err(err) => {
tracing::warn!(
method = %method,
path = %path,
elapsed_ms = started_at.elapsed().as_millis(),
peer_addr = peer_addr.as_deref().unwrap_or("-"),
error = %err,
"http request failed"
);
Err(err)
}
}
}
})
.route("/health", web::get().to(health))
.route("/metrics", web::get().to(metrics))
})
.bind(format!("0.0.0.0:{}", port))?;
Ok(srv.run())
}

36
app/email/src/main.rs Normal file
View File

@ -0,0 +1,36 @@
mod context;
mod health;
use context::AppContext;
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let ctx = AppContext::init()?;
tracing::info!("email service starting");
let health_port = ctx.config.email_health_port();
let health_server = health::start_health(health_port, ctx.metrics.clone())?;
let health_handle = health_server.handle();
let health_task = tokio::spawn(health_server);
tokio::select! {
result = email::EmailWorker::start_with_metrics(&ctx.config, Some(ctx.metrics.clone())) => {
if let Err(e) = result {
tracing::error!("email worker exited with error: {}", e);
}
}
_ = tokio::signal::ctrl_c() => {
tracing::info!("shutdown signal received, stopping email service");
health_handle.stop(true).await;
}
result = health_task => {
match result {
Ok(Ok(())) => tracing::info!("health server stopped"),
Ok(Err(e)) => tracing::error!("health server error: {}", e),
Err(e) => tracing::error!("health task panicked: {}", e),
}
}
}
Ok(())
}

52
app/gitdata/Cargo.toml Normal file
View File

@ -0,0 +1,52 @@
[package]
name = "app-gitdata"
version.workspace = true
edition.workspace = true
authors.workspace = true
description.workspace = true
repository.workspace = true
readme.workspace = true
homepage.workspace = true
license.workspace = true
keywords.workspace = true
categories.workspace = true
documentation.workspace = true
[[bin]]
name = "gitdata"
path = "src/main.rs"
[[bin]]
name = "gen-openapi"
path = "src/bin/gen-openapi.rs"
[dependencies]
anyhow = { workspace = true }
config = { workspace = true }
cache = { workspace = true }
db = { workspace = true }
service = { workspace = true }
session = { workspace = true }
api = { workspace = true }
email = { workspace = true }
storage = { workspace = true }
git = { workspace = true }
migrate = { workspace = true }
model = { workspace = true }
channel = { workspace = true }
socketio = { workspace = true }
tokio = { workspace = true, features = ["rt-multi-thread", "macros", "signal"] }
track = { workspace = true }
tracing = { workspace = true }
actix-web = { workspace = true, features = ["cookies", "secure-cookies"] }
tracing-actix-web = { workspace = true }
actix-ws = { workspace = true }
tonic = { workspace = true, features = ["transport"] }
deadpool-redis = { workspace = true }
redis = { workspace = true, features = ["cluster-async", "aio", "tokio-comp", "connection-manager", "cluster"] }
sqlx = { workspace = true, features = ["postgres", "runtime-tokio"] }
serde_json = { workspace = true }
uuid = { workspace = true, features = ["v4", "v7", "serde"] }
[lints]
workspace = true

View File

@ -0,0 +1,7 @@
use std::fs;
fn main() {
let json = api::openapi::openapi_json();
fs::write("openapi.json", json).expect("Failed to write openapi.json");
println!("openapi.json generated successfully");
}

141
app/gitdata/src/context.rs Normal file
View File

@ -0,0 +1,141 @@
use actix_web::cookie::Key;
use cache::{AppCache, AppCacheConfig};
use config::AppConfig;
use db::database::AppDatabase;
use deadpool_redis::{
PoolConfig, Runtime, Timeouts,
cluster::{Config, Pool as RedisPool},
};
use email::AppEmail;
use service::AppService;
use session::storage::RedisClusterSessionStore;
use storage::{AppStorage, AppStorageConfig};
use tonic::transport::Channel;
use channel::{CdnManager, ChannelBus, ChannelBusConfig};
use socketio::SocketIo;
pub struct AppContext {
pub config: AppConfig,
pub service: AppService,
pub session_store: RedisClusterSessionStore,
pub session_key: Key,
pub channel_bus: ChannelBus,
_otel_guard: Option<track::LgtmGuard>,
}
impl AppContext {
pub async fn init() -> anyhow::Result<Self> {
let config = AppConfig::load();
let otel_guard = init_tracing(&config)?;
tracing::info!("initializing database");
let mut db = AppDatabase::init(&config).await?;
tracing::info!("running database migrations");
migrate::run_up(db.writer()).await?;
tracing::info!("initializing cache");
let cache_config = AppCacheConfig::try_from(&config)?;
let mut cache = AppCache::init(cache_config).await?;
tracing::info!("initializing storage");
let storage_config = AppStorageConfig::try_from(&config)?;
let mut storage = AppStorage::init(storage_config).await?;
tracing::info!("initializing email");
let mut email = AppEmail::init(&config).await?;
tracing::info!("connecting to git RPC");
let rpc_addr = config.git_rpc_addr()?;
let rpc_port = config.git_rpc_port()?;
let git_channel =
Channel::from_shared(format!("http://{}:{}", rpc_addr, rpc_port))
.expect("invalid gRPC endpoint")
.connect()
.await?;
let metrics_registry = track::MetricsRegistry::new();
db.set_metrics(metrics_registry.clone());
cache.set_metrics(metrics_registry.clone());
storage.set_metrics(metrics_registry.clone());
email.set_metrics(metrics_registry.clone());
let service_metrics =
service::metrics::ServiceMetrics::init(&metrics_registry);
let service = AppService {
db,
cache,
email,
storage,
config: config.clone(),
git: git_channel,
redis_pool: init_redis_pool(&config)?,
metrics_registry,
metrics: service_metrics,
};
tracing::info!("initializing session store");
let redis_urls = config.redis_urls()?;
let session_store = RedisClusterSessionStore::new(redis_urls).await?;
tracing::info!("initializing session key");
let secret = config.session_secret()?;
let session_key = Key::from(secret.as_bytes());
tracing::info!("initializing channel bus");
let io = SocketIo::new();
let channel_config = ChannelBusConfig {
namespace: "/channel".to_owned(),
signing_secret: Some(secret.clone()),
..Default::default()
};
let cdn = CdnManager::new(service.storage.clone());
let channel_bus = ChannelBus::new(
service.db.clone(),
service.cache.clone(),
io,
channel_config,
cdn,
Some(service.metrics_registry.clone()),
);
channel_bus.attach().await?;
Ok(Self {
config,
service,
session_store,
session_key,
channel_bus,
_otel_guard: otel_guard,
})
}
}
fn init_tracing(
config: &AppConfig,
) -> anyhow::Result<Option<track::LgtmGuard>> {
track::init_lgtm(config)
}
fn init_redis_pool(config: &AppConfig) -> anyhow::Result<RedisPool> {
let redis_urls = config.redis_urls()?;
let pool_size = config.redis_pool_size()?;
let connect_timeout = config.redis_connect_timeout()?;
let acquire_timeout = config.redis_acquire_timeout()?;
let mut pool_config = PoolConfig::new(pool_size as usize);
pool_config.timeouts = Timeouts {
wait: Some(std::time::Duration::from_secs(acquire_timeout)),
create: Some(std::time::Duration::from_secs(connect_timeout)),
recycle: Some(std::time::Duration::from_secs(connect_timeout)),
};
let cfg = Config {
urls: Some(redis_urls),
connections: None,
pool: Some(pool_config),
read_from_replicas: false,
};
Ok(cfg.create_pool(Some(Runtime::Tokio1))?)
}

167
app/gitdata/src/main.rs Normal file
View File

@ -0,0 +1,167 @@
mod context;
mod shutdown;
use std::time::Instant;
use actix_web::{App, dev::Service};
use context::AppContext;
use service::ai::sync::spawn_model_sync_loop;
use tracing_actix_web::TracingLogger;
use track::{CounterVec, HistogramVec};
const REQUEST_LOG_EXCLUDED_PATHS: &[&str] = &[
"/health",
"/live",
"/ready",
"/metrics",
"/favicon.ico",
"/robots.txt",
];
fn should_log_request(path: &str) -> bool {
!REQUEST_LOG_EXCLUDED_PATHS.contains(&path)
}
fn record_http_request(
registry: &track::MetricsRegistry,
method: &str,
status: u16,
elapsed: std::time::Duration,
) {
http_requests_total(registry)
.with_label_values(&[method, &status.to_string()])
.inc();
http_request_duration(registry)
.with_label_values(&[method, &status.to_string()])
.observe(elapsed.as_secs_f64());
}
fn http_requests_total(registry: &track::MetricsRegistry) -> CounterVec {
registry
.register_counter_vec(
"http_requests_total",
"Total HTTP requests",
&["method", "status"],
)
.expect("failed to register http_requests_total")
}
fn http_request_duration(registry: &track::MetricsRegistry) -> HistogramVec {
registry
.register_histogram_vec(
"http_request_duration_seconds",
"HTTP request duration in seconds",
&["method", "status"],
vec![0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0],
)
.expect("failed to register http_request_duration_seconds")
}
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let ctx = AppContext::init().await?;
let api_port = ctx.config.api_port()?;
tracing::info!("GitDataAI API service starting on 0.0.0.0:{}", api_port);
let service = ctx.service.clone();
let session_store = ctx.session_store.clone();
let session_key = ctx.session_key.clone();
let channel_bus = ctx.channel_bus.clone();
let srv = actix_web::HttpServer::new(move || {
let session_middleware = session::SessionMiddleware::builder(
session_store.clone(),
session_key.clone(),
)
.cookie_secure(false)
.cookie_name("id".to_string())
.session_lifecycle(
session::config::PersistentSession::default()
.session_ttl(actix_web::cookie::time::Duration::days(30)),
)
.build();
let request_metrics = service.metrics_registry.clone();
App::new()
.app_data(actix_web::web::Data::new(service.clone()))
.app_data(actix_web::web::Data::new(channel_bus.clone()))
.wrap(TracingLogger::default())
.wrap_fn(move |req, srv| {
let metrics = request_metrics.clone();
let should_log = should_log_request(req.path());
let method = req.method().clone();
let path = req.path().to_owned();
let peer_addr =
req.connection_info().peer_addr().map(str::to_owned);
let started_at = Instant::now();
let fut = srv.call(req);
async move {
match fut.await {
Ok(res) => {
let elapsed = started_at.elapsed();
let status = res.status().as_u16();
record_http_request(
&metrics,
method.as_str(),
status,
elapsed,
);
if should_log {
tracing::info!(
method = %method,
path = %path,
status = status,
elapsed_ms = elapsed.as_millis(),
peer_addr = peer_addr.as_deref().unwrap_or("-"),
"http request"
);
}
Ok(res)
}
Err(err) => {
let elapsed = started_at.elapsed();
record_http_request(
&metrics,
method.as_str(),
500,
elapsed,
);
if should_log {
tracing::warn!(
method = %method,
path = %path,
elapsed_ms = elapsed.as_millis(),
peer_addr = peer_addr.as_deref().unwrap_or("-"),
error = %err,
"http request failed"
);
}
Err(err)
}
}
}
})
.wrap(session_middleware)
.configure(|cfg| api::configure(cfg, channel_bus.clone()))
})
.bind(format!("0.0.0.0:{}", api_port))?;
spawn_model_sync_loop(ctx.service.clone());
let server = srv.run();
tracing::info!("API server is running");
tokio::select! {
_ = server => {
tracing::info!("API server stopped");
}
_ = shutdown::wait_for_shutdown_signal() => {
tracing::info!("shutdown signal received, stopping gitdata API service");
}
}
Ok(())
}

View File

@ -0,0 +1,25 @@
pub async fn wait_for_shutdown_signal() {
let ctrl_c = async {
tokio::signal::ctrl_c()
.await
.expect("failed to listen for ctrl_c event");
};
#[cfg(unix)]
let terminate = async {
tokio::signal::unix::signal(
tokio::signal::unix::SignalKind::terminate(),
)
.expect("failed to listen for SIGTERM")
.recv()
.await;
};
#[cfg(not(unix))]
let terminate = std::future::pending::<()>();
tokio::select! {
_ = ctrl_c => {},
_ = terminate => {},
}
}

32
app/gitpod/Cargo.toml Normal file
View File

@ -0,0 +1,32 @@
[package]
name = "app-gitpod"
version.workspace = true
edition.workspace = true
authors.workspace = true
description.workspace = true
repository.workspace = true
readme.workspace = true
homepage.workspace = true
license.workspace = true
keywords.workspace = true
categories.workspace = true
documentation.workspace = true
[[bin]]
name = "gitpod"
path = "src/main.rs"
[dependencies]
anyhow = { workspace = true }
config = { workspace = true }
cache = { workspace = true }
db = { workspace = true }
git = { workspace = true }
tokio = { workspace = true, features = ["rt-multi-thread", "macros", "signal"] }
track = { workspace = true }
tracing = { workspace = true }
deadpool-redis = { workspace = true }
redis = { workspace = true, features = ["cluster-async", "aio", "tokio-comp"] }
[lints]
workspace = true

68
app/gitpod/src/context.rs Normal file
View File

@ -0,0 +1,68 @@
use std::time::Duration;
use cache::{AppCache, AppCacheConfig};
use config::AppConfig;
use db::database::AppDatabase;
use deadpool_redis::{PoolConfig, Runtime, Timeouts, cluster::Config};
pub struct AppContext {
pub config: AppConfig,
pub db: AppDatabase,
pub cache: AppCache,
pub redis_pool: deadpool_redis::cluster::Pool,
pub metrics: track::MetricsRegistry,
_otel_guard: Option<track::LgtmGuard>,
}
impl AppContext {
pub async fn init() -> anyhow::Result<Self> {
let config = AppConfig::load();
let otel_guard = init_tracing(&config)?;
tracing::info!("initializing database");
let mut db = AppDatabase::init(&config).await?;
tracing::info!("initializing cache");
let cache_config = AppCacheConfig::try_from(&config)?;
let mut cache = AppCache::init(cache_config).await?;
tracing::info!("initializing redis pool");
let redis_urls = config.redis_urls()?;
let pool_size = config.redis_pool_size()?;
let connect_timeout = config.redis_connect_timeout()?;
let acquire_timeout = config.redis_acquire_timeout()?;
let mut pool_config = PoolConfig::new(pool_size as usize);
pool_config.timeouts = Timeouts {
wait: Some(Duration::from_secs(acquire_timeout)),
create: Some(Duration::from_secs(connect_timeout)),
recycle: Some(Duration::from_secs(connect_timeout)),
};
let cfg = Config {
urls: Some(redis_urls),
connections: None,
pool: Some(pool_config),
read_from_replicas: false,
};
let redis_pool = cfg.create_pool(Some(Runtime::Tokio1))?;
let metrics = track::MetricsRegistry::new();
db.set_metrics(metrics.clone());
cache.set_metrics(metrics.clone());
Ok(Self {
config,
db,
cache,
redis_pool,
metrics,
_otel_guard: otel_guard,
})
}
}
fn init_tracing(
config: &AppConfig,
) -> anyhow::Result<Option<track::LgtmGuard>> {
track::init_lgtm(config)
}

134
app/gitpod/src/main.rs Normal file
View File

@ -0,0 +1,134 @@
mod context;
mod shutdown;
use context::AppContext;
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let ctx = AppContext::init().await?;
let http_port = ctx.config.git_http_port()?;
let ssh_port = ctx.config.ssh_port()?;
let rpc_addr = ctx.config.git_rpc_addr()?;
let rpc_port = ctx.config.git_rpc_port()?;
tracing::info!(
"gitpod service starting (HTTP:{} / SSH:{} / gRPC:{}:{})",
http_port,
ssh_port,
rpc_addr,
rpc_port
);
let metrics = ctx.metrics.clone();
let http_task = tokio::spawn(run_http_with_metrics(
ctx.config.clone(),
ctx.db.clone(),
ctx.cache.clone(),
ctx.redis_pool.clone(),
metrics.clone(),
));
let ssh_task = tokio::spawn(run_ssh_with_metrics(
ctx.config.clone(),
ctx.db.clone(),
ctx.cache.clone(),
ctx.redis_pool.clone(),
metrics.clone(),
));
let rpc_addr_parsed =
format!("{}:{}", rpc_addr, rpc_port).parse::<std::net::SocketAddr>()?;
let sync_service =
git::sync::ReceiveSyncService::new(ctx.redis_pool.clone());
let git_server = git::rpc::server::GitServer::new(
rpc_addr_parsed,
ctx.db.clone(),
ctx.cache.clone(),
sync_service,
);
let rpc_task = tokio::spawn(run_rpc_with_metrics(git_server, metrics));
tokio::select! {
result = http_task => {
match result {
Ok(Ok(())) => tracing::info!("HTTP server stopped"),
Ok(Err(e)) => tracing::error!("HTTP server error: {}", e),
Err(e) => tracing::error!("HTTP task panicked: {}", e),
}
}
result = ssh_task => {
match result {
Ok(Ok(())) => tracing::info!("SSH server stopped"),
Ok(Err(e)) => tracing::error!("SSH server error: {}", e),
Err(e) => tracing::error!("SSH task panicked: {}", e),
}
}
result = rpc_task => {
match result {
Ok(Ok(())) => tracing::info!("gRPC server stopped"),
Ok(Err(e)) => tracing::error!("gRPC server error: {}", e),
Err(e) => tracing::error!("gRPC task panicked: {}", e),
}
}
_ = shutdown::wait_for_shutdown_signal() => {
tracing::info!("shutdown signal received, stopping gitpod service");
}
}
Ok(())
}
async fn run_http_with_metrics(
config: config::AppConfig,
db: db::database::AppDatabase,
cache: cache::AppCache,
redis_pool: deadpool_redis::cluster::Pool,
metrics: track::MetricsRegistry,
) -> anyhow::Result<()> {
let result = git::http::run_http(
config,
db,
cache,
redis_pool,
Some(metrics.clone()),
)
.await;
let status = if result.is_ok() { "stop" } else { "error" };
gitpod_counter(&metrics, "gitpod_http_events_total", status);
result
}
async fn run_ssh_with_metrics(
config: config::AppConfig,
db: db::database::AppDatabase,
cache: cache::AppCache,
redis_pool: deadpool_redis::cluster::Pool,
metrics: track::MetricsRegistry,
) -> anyhow::Result<()> {
git::ssh::handler::set_ssh_metrics(metrics.clone());
let result = git::ssh::run_ssh(config, db, cache, redis_pool).await;
let status = if result.is_ok() { "stop" } else { "error" };
gitpod_counter(&metrics, "gitpod_ssh_events_total", status);
result
}
async fn run_rpc_with_metrics(
server: git::rpc::server::GitServer,
metrics: track::MetricsRegistry,
) -> anyhow::Result<()> {
let result = server.serve().await.map_err(|e| anyhow::anyhow!("{}", e));
let status = if result.is_ok() { "stop" } else { "error" };
gitpod_counter(&metrics, "gitpod_rpc_events_total", status);
result
}
fn gitpod_counter(registry: &track::MetricsRegistry, name: &str, label: &str) {
if let Ok(cv) = registry.register_counter_vec(
name,
"Gitpod server lifecycle events",
&["event"],
) {
cv.with_label_values(&[label]).inc();
}
}

View File

@ -0,0 +1,25 @@
pub async fn wait_for_shutdown_signal() {
let ctrl_c = async {
tokio::signal::ctrl_c()
.await
.expect("failed to listen for ctrl_c event");
};
#[cfg(unix)]
let terminate = async {
tokio::signal::unix::signal(
tokio::signal::unix::SignalKind::terminate(),
)
.expect("failed to listen for SIGTERM")
.recv()
.await;
};
#[cfg(not(unix))]
let terminate = std::future::pending::<()>();
tokio::select! {
_ = ctrl_c => {},
_ = terminate => {},
}
}

37
app/gitsync/Cargo.toml Normal file
View File

@ -0,0 +1,37 @@
[package]
name = "app-gitsync"
version.workspace = true
edition.workspace = true
authors.workspace = true
description.workspace = true
repository.workspace = true
readme.workspace = true
homepage.workspace = true
license.workspace = true
keywords.workspace = true
categories.workspace = true
documentation.workspace = true
[[bin]]
name = "gitsync"
path = "src/main.rs"
[dependencies]
anyhow = { workspace = true }
config = { workspace = true }
cache = { workspace = true }
db = { workspace = true }
git = { workspace = true }
tokio = { workspace = true, features = ["rt-multi-thread", "macros", "signal"] }
track = { workspace = true }
tracing = { workspace = true }
actix-web = { workspace = true }
tracing-actix-web = { workspace = true }
deadpool-redis = { workspace = true }
redis = { workspace = true, features = ["cluster-async", "aio", "tokio-comp"] }
uuid = { workspace = true }
serde_json = { workspace = true }
sqlx = { workspace = true, features = ["postgres", "runtime-tokio"] }
[lints]
workspace = true

View File

@ -0,0 +1,68 @@
use std::time::Duration;
use cache::{AppCache, AppCacheConfig};
use config::AppConfig;
use db::database::AppDatabase;
use deadpool_redis::{PoolConfig, Runtime, Timeouts, cluster::Config};
pub struct AppContext {
pub config: AppConfig,
pub db: AppDatabase,
pub cache: AppCache,
pub redis_pool: deadpool_redis::cluster::Pool,
pub metrics: track::MetricsRegistry,
_otel_guard: Option<track::LgtmGuard>,
}
impl AppContext {
pub async fn init() -> anyhow::Result<Self> {
let config = AppConfig::load();
let otel_guard = init_tracing(&config)?;
tracing::info!("initializing database");
let mut db = AppDatabase::init(&config).await?;
tracing::info!("initializing cache");
let cache_config = AppCacheConfig::try_from(&config)?;
let mut cache = AppCache::init(cache_config).await?;
tracing::info!("initializing redis pool");
let redis_urls = config.redis_urls()?;
let pool_size = config.redis_pool_size()?;
let connect_timeout = config.redis_connect_timeout()?;
let acquire_timeout = config.redis_acquire_timeout()?;
let mut pool_config = PoolConfig::new(pool_size as usize);
pool_config.timeouts = Timeouts {
wait: Some(Duration::from_secs(acquire_timeout)),
create: Some(Duration::from_secs(connect_timeout)),
recycle: Some(Duration::from_secs(connect_timeout)),
};
let cfg = Config {
urls: Some(redis_urls),
connections: None,
pool: Some(pool_config),
read_from_replicas: false,
};
let redis_pool = cfg.create_pool(Some(Runtime::Tokio1))?;
let metrics = track::MetricsRegistry::new();
db.set_metrics(metrics.clone());
cache.set_metrics(metrics.clone());
Ok(Self {
config,
db,
cache,
redis_pool,
metrics,
_otel_guard: otel_guard,
})
}
}
fn init_tracing(
config: &AppConfig,
) -> anyhow::Result<Option<track::LgtmGuard>> {
track::init_lgtm(config)
}

115
app/gitsync/src/health.rs Normal file
View File

@ -0,0 +1,115 @@
use std::time::Instant;
use actix_web::dev::Service;
use actix_web::{App, HttpResponse, HttpServer, dev::Server, web};
use cache::AppCache;
use db::database::AppDatabase;
use tracing_actix_web::TracingLogger;
const REQUEST_LOG_EXCLUDED_PATHS: &[&str] = &[
"/health",
"/live",
"/ready",
"/metrics",
"/favicon.ico",
"/robots.txt",
];
fn should_log_request(path: &str) -> bool {
!REQUEST_LOG_EXCLUDED_PATHS.contains(&path)
}
async fn health(
db: web::Data<AppDatabase>,
cache: web::Data<AppCache>,
) -> HttpResponse {
let db_ok = sqlx::query("SELECT 1").execute(db.reader()).await.is_ok();
let cache_ok = cache.ping_cluster().await.is_ok();
if db_ok && cache_ok {
HttpResponse::Ok().json(serde_json::json!({
"status": "ok",
"db": "ok",
"cache": "ok",
}))
} else {
HttpResponse::ServiceUnavailable().json(serde_json::json!({
"status": "unhealthy",
"db": if db_ok { "ok" } else { "error" },
"cache": if cache_ok { "ok" } else { "error" },
}))
}
}
async fn metrics(metrics: web::Data<track::MetricsRegistry>) -> HttpResponse {
match metrics.encode() {
Ok(body) => HttpResponse::Ok()
.content_type("text/plain; version=0.0.4")
.body(body),
Err(e) => HttpResponse::InternalServerError()
.content_type("text/plain")
.body(format!("metrics encoding error: {e}")),
}
}
pub fn start_health(
port: u16,
db: AppDatabase,
cache: AppCache,
metrics_registry: track::MetricsRegistry,
) -> anyhow::Result<Server> {
tracing::info!("health endpoint starting on 0.0.0.0:{}", port);
let srv = HttpServer::new(move || {
App::new()
.app_data(web::Data::new(db.clone()))
.app_data(web::Data::new(cache.clone()))
.app_data(web::Data::new(metrics_registry.clone()))
.wrap(TracingLogger::default())
.wrap_fn(|req, srv| {
let should_log = should_log_request(req.path());
let method = req.method().clone();
let path = req.path().to_owned();
let peer_addr =
req.connection_info().peer_addr().map(str::to_owned);
let started_at = Instant::now();
let fut = srv.call(req);
async move {
match fut.await {
Ok(res) => {
if should_log {
tracing::info!(
method = %method,
path = %path,
status = res.status().as_u16(),
elapsed_ms = started_at.elapsed().as_millis(),
peer_addr = peer_addr.as_deref().unwrap_or("-"),
"http request"
);
}
Ok(res)
}
Err(err) => {
if should_log {
tracing::warn!(
method = %method,
path = %path,
elapsed_ms = started_at.elapsed().as_millis(),
peer_addr = peer_addr.as_deref().unwrap_or("-"),
error = %err,
"http request failed"
);
}
Err(err)
}
}
}
})
.route("/health", web::get().to(health))
.route("/metrics", web::get().to(metrics))
})
.bind(format!("0.0.0.0:{}", port))?;
Ok(srv.run())
}

70
app/gitsync/src/main.rs Normal file
View File

@ -0,0 +1,70 @@
mod context;
mod health;
mod shutdown;
use context::AppContext;
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let ctx = AppContext::init().await?;
tracing::info!("gitsync service starting");
let health_port = ctx.config.gitsync_health_port();
let health_server = health::start_health(
health_port,
ctx.db.clone(),
ctx.cache.clone(),
ctx.metrics.clone(),
)?;
let health_handle = health_server.handle();
let health_task = tokio::spawn(health_server);
let sync_service =
git::sync::ReceiveSyncService::new(ctx.redis_pool.clone());
let consumer = git::sync::consumer::SyncConsumer::new(sync_service, 5);
let mut worker = git::sync::worker::SyncWorker::new(
consumer,
ctx.db.clone(),
ctx.cache.clone(),
ctx.redis_pool.clone(),
ctx.config.clone(),
format!("gitsync-{}", uuid::Uuid::new_v4()),
);
worker.set_metrics(ctx.metrics.clone());
let metrics = ctx.metrics.clone();
let worker_task = tokio::spawn(async move {
worker.run().await;
record_sync_event(&metrics, "worker_stopped");
});
tokio::select! {
result = health_task => {
match result {
Ok(Ok(())) => tracing::info!("health server stopped"),
Ok(Err(e)) => tracing::error!("health server error: {}", e),
Err(e) => tracing::error!("health task panicked: {}", e),
}
}
_ = worker_task => {
tracing::info!("sync worker stopped");
}
_ = shutdown::wait_for_shutdown_signal() => {
tracing::info!("shutdown signal received, stopping gitsync service");
health_handle.stop(true).await;
}
}
Ok(())
}
fn record_sync_event(registry: &track::MetricsRegistry, event: &str) {
if let Ok(cv) = registry.register_counter_vec(
"gitsync_worker_events_total",
"Gitsync worker lifecycle events",
&["event"],
) {
cv.with_label_values(&[event]).inc();
}
}

View File

@ -0,0 +1,25 @@
pub async fn wait_for_shutdown_signal() {
let ctrl_c = async {
tokio::signal::ctrl_c()
.await
.expect("failed to listen for ctrl_c event");
};
#[cfg(unix)]
let terminate = async {
tokio::signal::unix::signal(
tokio::signal::unix::SignalKind::terminate(),
)
.expect("failed to listen for SIGTERM")
.recv()
.await;
};
#[cfg(not(unix))]
let terminate = std::future::pending::<()>();
tokio::select! {
_ = ctrl_c => {},
_ = terminate => {},
}
}

View File

@ -1,39 +0,0 @@
[package]
name = "app"
version.workspace = true
edition.workspace = true
authors.workspace = true
description.workspace = true
repository.workspace = true
readme.workspace = true
homepage.workspace = true
license.workspace = true
keywords.workspace = true
categories.workspace = true
documentation.workspace = true
[dependencies]
tokio = { workspace = true, features = ["full"] }
uuid = { workspace = true }
service = { workspace = true }
observability = { workspace = true }
room = { workspace = true }
sha2 = { workspace = true }
hkdf = { workspace = true }
hmac = { workspace = true }
api = { workspace = true }
session = { workspace = true }
config = { workspace = true }
db = { workspace = true }
migrate = { workspace = true }
actix-web = { workspace = true }
actix-cors = { workspace = true }
futures = { workspace = true }
tracing = { workspace = true }
anyhow = { workspace = true }
clap = { workspace = true }
sea-orm = { workspace = true }
serde_json = { workspace = true }
chrono = { workspace = true }
[lints]
workspace = true

View File

@ -1,12 +0,0 @@
use clap::Parser;
#[derive(Parser, Debug)]
#[command(name = "app")]
#[command(version)]
pub struct ServerArgs {
#[arg(long, short)]
pub bind: Option<String>,
#[arg(long)]
pub workers: Option<usize>,
}

View File

@ -1,133 +0,0 @@
//! Structured HTTP request logging middleware using tracing.
//!
//! Logs every incoming request with method, path, status code,
//! response time, client IP, authenticated user ID, and trace_id.
use actix_web::dev::{Service, ServiceRequest, ServiceResponse, Transform};
use futures::future::{LocalBoxFuture, Ready, ok};
use session::SessionExt;
use std::sync::Arc;
use std::task::{Context, Poll};
use std::time::Instant;
use uuid::Uuid;
/// Default log format: `{method} {path} {status} {duration_ms}ms`
pub struct RequestLogger {
trace_id_header: String,
}
impl RequestLogger {
pub fn new(trace_id_header: String) -> Self {
Self { trace_id_header }
}
}
impl<S, B> Transform<S, ServiceRequest> for RequestLogger
where
S: Service<ServiceRequest, Response = ServiceResponse<B>, Error = actix_web::Error> + 'static,
S::Future: 'static,
B: 'static,
{
type Response = ServiceResponse<B>;
type Error = actix_web::Error;
type Transform = RequestLoggerMiddleware<S>;
type InitError = ();
type Future = Ready<Result<Self::Transform, Self::InitError>>;
fn new_transform(&self, service: S) -> Self::Future {
ok(RequestLoggerMiddleware {
service: Arc::new(service),
trace_id_header: self.trace_id_header.clone(),
})
}
}
pub struct RequestLoggerMiddleware<S> {
service: Arc<S>,
trace_id_header: String,
}
impl<S> Clone for RequestLoggerMiddleware<S> {
fn clone(&self) -> Self {
Self {
service: self.service.clone(),
trace_id_header: self.trace_id_header.clone(),
}
}
}
impl<S, B> Service<ServiceRequest> for RequestLoggerMiddleware<S>
where
S: Service<ServiceRequest, Response = ServiceResponse<B>, Error = actix_web::Error> + 'static,
S::Future: 'static,
B: 'static,
{
type Response = ServiceResponse<B>;
type Error = actix_web::Error;
type Future = LocalBoxFuture<'static, Result<Self::Response, Self::Error>>;
fn poll_ready(&self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
self.service.poll_ready(cx)
}
fn call(&self, req: ServiceRequest) -> Self::Future {
let started = Instant::now();
let trace_id_header = self.trace_id_header.clone();
let method = req.method().to_string();
let path = req.path().to_string();
let query = req.query_string().to_string();
let remote = req
.connection_info()
.realip_remote_addr()
.map(|s| s.to_string())
.unwrap_or_else(|| "unknown".to_string());
let user_id: Option<Uuid> = req.get_session().user();
let trace_id = Uuid::now_v7().to_string();
let full_path = if query.is_empty() {
path.clone()
} else {
format!("{}?{}", path, query)
};
let service = self.service.clone();
Box::pin(async move {
let res = service.call(req).await?;
let elapsed = started.elapsed();
let status = res.status();
let status_code = status.as_u16();
let is_health = path == "/health";
if !is_health {
let user_id_str = user_id
.map(|u: Uuid| u.to_string())
.unwrap_or_else(|| "-".to_string());
let duration_ms = elapsed.as_millis() as u64;
let log_args = (
method = %method,
path = %full_path,
status = status_code,
duration_ms = duration_ms,
remote = %remote,
user_id = %user_id_str,
trace_id = %trace_id,
);
match status_code {
200..=299 => {
tracing::info!(log_args, "http_request");
}
400..=499 => {
tracing::warn!(log_args, "http_request");
}
_ => {
tracing::error!(log_args, "http_request");
}
}
}
Ok(res)
})
}
}

View File

@ -1,350 +0,0 @@
use actix_cors::Cors;
use actix_web::cookie::time::Duration;
use actix_web::dev::{Service, ServiceRequest, ServiceResponse};
use actix_web::{App, HttpResponse, HttpServer, cookie::Key, web};
use api::{robots, sidemap};
use clap::Parser;
use db::cache::AppCache;
use db::database::AppDatabase;
use futures::future::LocalBoxFuture;
use observability::{
HttpMetrics, HttpSnapshotGuard, MetricsMiddleware, TracingSpanMiddleware,
init_tracing_subscriber, install_recorder, prometheus_handler, push::MetricsPusher,
spawn_http_metrics_poller,
};
use sea_orm::ConnectionTrait;
use service::AppService;
use session::SessionMiddleware;
use session::config::{PersistentSession, SessionLifecycle, TtlExtensionPolicy};
use session::storage::RedisClusterSessionStore;
use std::sync::Arc;
use std::task::{Context, Poll};
use std::time::Instant;
mod args;
use args::ServerArgs;
use config::AppConfig;
use migrate::{Migrator, MigratorTrait};
#[derive(Clone)]
pub struct AppState {
pub db: AppDatabase,
pub cache: AppCache,
}
/// Custom middleware that logs requests except for noisy paths.
struct RequestLogger;
impl<S, B> actix_web::dev::Transform<S, ServiceRequest> for RequestLogger
where
S: Service<ServiceRequest, Response = ServiceResponse<B>, Error = actix_web::Error>,
S::Future: 'static,
B: 'static,
{
type Response = ServiceResponse<B>;
type Error = actix_web::Error;
type Transform = RequestLoggerService<S>;
type InitError = ();
type Future = futures::future::Ready<Result<Self::Transform, Self::InitError>>;
fn new_transform(&self, service: S) -> Self::Future {
futures::future::ok(RequestLoggerService {
service,
_marker: std::marker::PhantomData,
})
}
}
struct RequestLoggerService<S> {
service: S,
_marker: std::marker::PhantomData<fn(ServiceRequest)>,
}
impl<S, B> actix_web::dev::Service<ServiceRequest> for RequestLoggerService<S>
where
S: actix_web::dev::Service<
ServiceRequest,
Response = ServiceResponse<B>,
Error = actix_web::Error,
>,
S::Future: 'static,
B: 'static,
{
type Response = ServiceResponse<B>;
type Error = actix_web::Error;
type Future = LocalBoxFuture<'static, Result<Self::Response, Self::Error>>;
fn poll_ready(&self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
self.service.poll_ready(cx)
}
fn call(&self, req: ServiceRequest) -> Self::Future {
let path = req.path().to_string();
let method = req.method().to_string();
let should_log = !(path == "/health"
|| path == "/metrics"
|| path.starts_with("/ws")
|| path.starts_with("/assets"));
let start = Instant::now();
let fut = self.service.call(req);
Box::pin(async move {
let res = fut.await?;
if should_log {
tracing::info!(
target: "http_request",
method = %method,
path = %path,
status = res.status().as_u16(),
elapsed = ?start.elapsed(),
"{} {} {} {:?}",
method,
path,
res.status().as_u16(),
start.elapsed()
);
}
Ok(res)
})
}
}
fn build_session_key(cfg: &AppConfig) -> anyhow::Result<Key> {
if let Some(secret) = cfg.env.get("APP_SESSION_SECRET") {
if secret.len() < 32 {
tracing::warn!(
secret_len = secret.len(),
"APP_SESSION_SECRET is too short (<32 bytes), using generated key instead"
);
return Ok(Key::generate());
}
use hkdf::Hkdf;
use sha2::Sha256;
// HKDF-SHA256: standard key derivation with info string for domain separation
let hk = Hkdf::<Sha256>::new(Some(b"session-cookie-key"), secret.as_bytes());
let mut okm = [0u8; 64];
hk.expand(b"actix-session-signing-key", &mut okm)
.map_err(|e| anyhow::anyhow!("HKDF expand failed: {}", e))?;
return Ok(Key::from(&okm));
}
tracing::warn!(
"APP_SESSION_SECRET not set, using generated key (sessions invalidated on restart)"
);
Ok(Key::generate())
}
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let cfg = AppConfig::load();
let log_level = cfg.log_level().unwrap_or_else(|_| "info".to_string());
let otel_enabled = cfg.otel_enabled().unwrap_or(false);
init_tracing_subscriber(&log_level, false);
tracing::info!(
app_name = %cfg.app_name().unwrap_or_default(),
app_version = %cfg.app_version().unwrap_or_default(),
"Starting application"
);
let db = AppDatabase::init(&cfg).await?;
tracing::info!("Database connected");
let redis_urls = cfg.redis_urls()?;
let store: RedisClusterSessionStore = RedisClusterSessionStore::new(redis_urls).await?;
tracing::info!("Redis connected");
let cache = AppCache::init(&cfg).await?;
tracing::info!("Cache initialized");
run_migrations(&db).await?;
let session_key = build_session_key(&cfg)?;
let args = ServerArgs::parse();
let service = AppService::new(cfg.clone()).await?;
tracing::info!("AppService initialized");
let _model_sync_handle = service.clone().start_sync_task();
// TODO: workspace module not yet wired — billing alert task pending
// let _billing_alert_handle = service.clone().start_billing_alert_task();
let (shutdown_tx, shutdown_rx) = tokio::sync::broadcast::channel::<()>(1);
let worker_service = service.clone();
let worker_handle =
tokio::spawn(async move { worker_service.start_room_workers(shutdown_rx).await });
let _otel_guard = if otel_enabled {
let endpoint = cfg
.otel_endpoint()
.unwrap_or_else(|_| "http://localhost:4317".to_string());
let service_name = cfg
.otel_service_name()
.unwrap_or_else(|_| "app".to_string());
let service_version = cfg
.otel_service_version()
.unwrap_or_else(|_| "0.1.0".to_string());
tracing::info!(endpoint = %endpoint, service = %service_name, "OTLP tracing enabled");
let guard =
observability::init_otlp(&endpoint, &service_name, &service_version, &log_level)
.map_err(|e| anyhow::anyhow!("OTLP init failed: {}", e))?;
guard
} else {
None
};
let prometheus_handle = install_recorder();
let prometheus_handle_data = web::Data::new(prometheus_handle.clone());
let http_metrics = std::sync::Arc::new(HttpMetrics::new());
let http_snapshot: HttpSnapshotGuard = std::sync::Arc::new(std::sync::RwLock::new(
observability::HttpMetricsSnapshot::default(),
));
let http_snapshot_for_poller = http_snapshot.clone();
spawn_http_metrics_poller(
http_metrics.clone(),
http_snapshot_for_poller,
std::time::Duration::from_secs(15),
);
let http_snapshot_data = web::Data::new(http_snapshot);
// Metrics pusher: periodically push all metrics to apps/metrics aggregator
if let Some(push_url) = std::env::var("METRICS_PUSH_URL").ok() {
let pusher = MetricsPusher::new(&push_url, "app");
pusher.spawn(
http_metrics.clone(),
Arc::new(prometheus_handle.clone()),
std::time::Duration::from_secs(15),
);
tracing::info!(push_url = %push_url, "Metrics pusher started (interval 15s)");
}
let bind_addr = args.bind.unwrap_or_else(|| "127.0.0.1:8080".to_string());
tracing::info!(bind_addr = %bind_addr, "Listening");
let http_metrics_server = http_metrics.clone();
let cors_origins: Vec<String> = cfg
.env
.get("CORS_ORIGINS")
.map(|s| {
s.split(',')
.map(|s| s.trim().to_string())
.filter(|s| !s.is_empty())
.collect()
})
.unwrap_or_else(|| vec!["http://localhost:5173".to_string()]);
let cookie_secure = cfg
.env
.get("APP_COOKIE_SECURE")
.map(|s| s != "false")
.unwrap_or(true);
tracing::info!(cookie_secure = cookie_secure, "Cookie secure mode");
HttpServer::new(move || {
let mut cors = Cors::default();
for origin in &cors_origins {
cors = cors.allowed_origin(origin);
}
let cors = cors
.allowed_methods(["GET", "POST", "PUT", "PATCH", "DELETE", "OPTIONS"])
.allowed_headers([
"Content-Type",
"Authorization",
"X-Requested-With",
"Accept",
"Origin",
])
.supports_credentials()
.max_age(3600);
let security_headers = actix_web::middleware::DefaultHeaders::new()
.add(("X-Content-Type-Options", "nosniff"))
.add(("X-Frame-Options", "DENY"))
.add(("Referrer-Policy", "strict-origin-when-cross-origin"));
let session_mw = SessionMiddleware::builder(store.clone(), session_key.clone())
.cookie_name("id".to_string())
.cookie_path("/".to_string())
.cookie_secure(cookie_secure)
.cookie_http_only(true)
.session_lifecycle(SessionLifecycle::PersistentSession(
PersistentSession::default()
.session_ttl(Duration::days(30))
.session_ttl_extension_policy(TtlExtensionPolicy::OnEveryRequest),
))
.build();
let metrics_mw = MetricsMiddleware::new(http_metrics_server.clone());
App::new()
.wrap(cors)
.wrap(security_headers)
.wrap(session_mw)
.wrap(RequestLogger)
.wrap(metrics_mw)
.wrap(TracingSpanMiddleware::new())
.app_data(web::Data::new(AppState {
db: db.clone(),
cache: cache.clone(),
}))
.app_data(web::Data::new(service.clone()))
.app_data(web::Data::new(cfg.clone()))
.app_data(web::Data::new(db.clone()))
.app_data(web::Data::new(cache.clone()))
.app_data(http_snapshot_data.clone())
.app_data(prometheus_handle_data.clone())
.route("/robots.txt", web::get().to(robots::robots))
.route("/sitemap.xml", web::get().to(sidemap::sitemap))
.service(
web::scope("/sidemap")
.route("", web::get().to(sidemap::sitemap))
.route("/static", web::get().to(sidemap::sitemap_static))
.route("/users", web::get().to(sidemap::sitemap_users))
.route("/projects", web::get().to(sidemap::sitemap_projects))
.route("/repos", web::get().to(sidemap::sitemap_repos)),
)
.route("/health", web::get().to(health_check))
.route("/metrics", web::get().to(prometheus_handler))
.configure(api::route::init_routes)
})
.bind(&bind_addr)?
.run()
.await?;
tracing::info!("Server stopped, shutting down room workers");
let _ = shutdown_tx.send(());
let _ = worker_handle.await;
tracing::info!("Room workers stopped");
Ok(())
}
async fn run_migrations(db: &AppDatabase) -> anyhow::Result<()> {
tracing::info!("Running database migrations...");
Migrator::up(db.writer(), None)
.await
.map_err(|e| anyhow::anyhow!("Migration failed: {:?}", e))?;
tracing::info!("Migrations completed");
Ok(())
}
async fn health_check(state: web::Data<AppState>) -> HttpResponse {
let db_ok = db_ping(&state.db).await;
let cache_ok = cache_ping(&state.cache).await;
let healthy = db_ok && cache_ok;
if healthy {
HttpResponse::Ok().json(serde_json::json!({
"status": "ok",
"db": "ok",
"cache": "ok",
}))
} else {
HttpResponse::ServiceUnavailable().json(serde_json::json!({
"status": "unhealthy",
"db": if db_ok { "ok" } else { "error" },
"cache": if cache_ok { "ok" } else { "error" },
}))
}
}
async fn db_ping(db: &AppDatabase) -> bool {
let writer_ok = db.writer().execute_unprepared("SELECT 1").await.is_ok();
let reader_ok = db.reader().execute_unprepared("SELECT 1").await.is_ok();
writer_ok && reader_ok
}
async fn cache_ping(cache: &AppCache) -> bool {
cache.conn().await.is_ok()
}

View File

@ -1,36 +0,0 @@
[package]
name = "email-server"
version.workspace = true
edition.workspace = true
authors.workspace = true
description.workspace = true
repository.workspace = true
readme.workspace = true
homepage.workspace = true
license.workspace = true
keywords.workspace = true
categories.workspace = true
documentation.workspace = true
[[bin]]
name = "email-worker"
path = "src/main.rs"
[dependencies]
tokio = { workspace = true, features = ["full"] }
service = { workspace = true }
db = { workspace = true }
config = { workspace = true }
tracing = { workspace = true }
observability = { workspace = true }
anyhow = { workspace = true }
clap = { workspace = true, features = ["derive"] }
chrono = { workspace = true, features = ["serde"] }
hyper = { workspace = true }
serde_json = { workspace = true }
sea-orm = { workspace = true }
metrics = "0.22"
metrics-exporter-prometheus = "0.13"
[lints]
workspace = true

View File

@ -1,165 +0,0 @@
use clap::Parser;
use config::AppConfig;
use metrics::{Unit, describe_counter};
use metrics_exporter_prometheus::PrometheusHandle;
use observability::{HttpMetrics, init_tracing_subscriber, install_recorder, push::MetricsPusher};
use sea_orm::ConnectionTrait;
use service::AppService;
use std::sync::Arc;
#[derive(Parser, Debug)]
#[command(name = "email-worker")]
#[command(version)]
struct Args {
#[arg(long, default_value = "info")]
log_level: String,
}
async fn http_handler(
db: Arc<db::database::AppDatabase>,
cache: Arc<db::cache::AppCache>,
metrics: Arc<PrometheusHandle>,
req: hyper::Request<hyper::Body>,
) -> Result<hyper::Response<hyper::Body>, std::convert::Infallible> {
match req.uri().path() {
"/health" => {
let writer_ok = db.writer().execute_unprepared("SELECT 1").await.is_ok();
let reader_ok = db.reader().execute_unprepared("SELECT 1").await.is_ok();
let db_ok = writer_ok && reader_ok;
let cache_ok = cache.conn().await.is_ok();
let body = serde_json::json!({
"status": if db_ok && cache_ok { "ok" } else { "unhealthy" },
"db": if db_ok { "ok" } else { "error" },
"cache": if cache_ok { "ok" } else { "error" },
});
let status = if db_ok && cache_ok { 200 } else { 503 };
let body_bytes = match serde_json::to_string(&body) {
Ok(s) => hyper::Body::from(s),
Err(e) => {
return Ok(hyper::Response::builder()
.status(500)
.body(hyper::Body::from(format!("serialize error: {}", e)))
.expect("static response"));
}
};
Ok(hyper::Response::builder()
.status(status)
.header("content-type", "application/json")
.body(body_bytes)
.expect("static response"))
}
"/metrics" => {
let body = metrics.render();
Ok(hyper::Response::builder()
.status(200)
.header("content-type", "text/plain; version=0.0.4; charset=utf-8")
.body(hyper::Body::from(body))
.unwrap())
}
_ => Ok(hyper::Response::builder()
.status(404)
.body(hyper::Body::from("not found"))
.unwrap()),
}
}
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let args = Args::parse();
let cfg = AppConfig::load();
init_tracing_subscriber(&args.log_level, false);
// Pre-register all email/queue metrics so they appear in /metrics even before first event.
describe_counter!(
"email_queued_total",
Unit::Count,
"Emails written to Redis stream"
);
describe_counter!(
"email_consumed_total",
Unit::Count,
"Emails consumed from queue"
);
describe_counter!(
"email_batch_size",
Unit::Count,
"Email consumer batch sizes accumulated"
);
describe_counter!(
"email_validation_skipped_total",
Unit::Count,
"Emails skipped due to invalid recipient"
);
describe_counter!(
"email_build_errors_total",
Unit::Count,
"Email message build failures"
);
describe_counter!(
"email_send_attempts_total",
Unit::Count,
"SMTP send attempts (including retries)"
);
describe_counter!("email_sent_total", Unit::Count, "Emails sent successfully");
describe_counter!(
"email_send_failures_total",
Unit::Count,
"Emails that failed after all retries"
);
let metrics_handle = Arc::new(install_recorder());
let http_metrics = Arc::new(HttpMetrics::new()); // Worker app — HTTP section will be empty
// Metrics pusher: periodically push all metrics to apps/metrics aggregator
if let Some(push_url) = std::env::var("METRICS_PUSH_URL").ok() {
let pusher = MetricsPusher::new(&push_url, "email");
pusher.spawn(
http_metrics.clone(),
metrics_handle.clone(),
std::time::Duration::from_secs(15),
);
tracing::info!(push_url = %push_url, "Metrics pusher started (interval 15s)");
}
tracing::info!("Starting email worker");
let service = AppService::new(cfg).await?;
let db = Arc::new(service.db.clone());
let cache = Arc::new(service.cache.clone());
let (shutdown_tx, shutdown_rx) = tokio::sync::broadcast::channel::<()>(1);
tokio::spawn(async move {
tokio::signal::ctrl_c().await.ok();
tracing::info!("shutting down email worker");
let _ = shutdown_tx.send(());
});
// Start health/metrics server on a dedicated port
let health_db = db.clone();
let health_cache = cache.clone();
let health_metrics = metrics_handle.clone();
let health_addr: std::net::SocketAddr = ([0, 0, 0, 0], 8084).into();
let health_service = hyper::service::make_service_fn(move |_| {
let db = health_db.clone();
let cache = health_cache.clone();
let metrics = health_metrics.clone();
let service = hyper::service::service_fn(move |req| {
http_handler(db.clone(), cache.clone(), metrics.clone(), req)
});
async move { Ok::<_, std::convert::Infallible>(service) }
});
let health_server = hyper::Server::bind(&health_addr).serve(health_service);
tracing::info!(port = 8084, "health/metrics server started");
tokio::spawn(async move {
if let Err(e) = health_server.await {
tracing::error!("health check server error: {}", e);
}
});
service.start_email_workers(shutdown_rx).await?;
tracing::info!("email worker stopped");
Ok(())
}

View File

@ -1,46 +0,0 @@
[package]
name = "gingress"
version.workspace = true
edition.workspace = true
authors.workspace = true
description = "GIngress control plane: Kubernetes Ingress Controller using kube-rs"
repository.workspace = true
readme.workspace = true
homepage.workspace = true
license.workspace = true
keywords.workspace = true
categories.workspace = true
documentation.workspace = true
[[bin]]
name = "gingress"
path = "src/main.rs"
[[bin]]
name = "kubectl-gingress"
path = "src/bin/kubectl-gingress/main.rs"
[dependencies]
gingress-proxy = { workspace = true }
kube = { workspace = true }
k8s-openapi = { workspace = true }
tokio = { workspace = true, features = ["full"] }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
serde_yaml = { workspace = true }
tracing = { workspace = true }
observability = { workspace = true }
anyhow = { workspace = true }
thiserror = { workspace = true }
dashmap = { workspace = true }
futures-util = { workspace = true }
futures = { workspace = true }
clap = { workspace = true }
url = { workspace = true }
x509-parser = "0.17"
rustls-pemfile = "2"
[lints]
workspace = true

View File

@ -1,887 +0,0 @@
//! kubectl-gingress — kubectl plugin for managing GIngress resources.
//!
//! Usage (via kubectl): kubectl gingress <subcommand>
//! Usage (standalone): kubectl-gingress <subcommand>
use clap::{Parser, Subcommand};
use k8s_openapi::api::core::v1::{Pod, Secret};
use k8s_openapi::api::networking::v1::{HTTPIngressPath, Ingress};
use kube::api::ListParams;
use kube::{Api, Client, ResourceExt};
const INGRESS_CLASS: &str = "gingress";
#[derive(Parser)]
#[command(
name = "kubectl-gingress",
bin_name = "kubectl gingress",
about = "Manage GIngress — Kubernetes Ingress Controller",
version
)]
struct Cli {
#[command(subcommand)]
command: Command,
}
#[derive(Subcommand)]
enum Command {
/// List all Ingress resources managed by GIngress
#[command(alias = "ls")]
List {
/// Filter by namespace (omit for all namespaces)
#[arg(short, long)]
namespace: Option<String>,
/// Output as JSON
#[arg(long)]
json: bool,
},
/// Show the routing table (host → path → backend)
Routes {
/// Filter by namespace
#[arg(short, long)]
namespace: Option<String>,
/// Filter by host
#[arg(short = 'H', long)]
host: Option<String>,
/// Output as JSON
#[arg(long)]
json: bool,
},
/// Show backend services and their endpoints
Backends {
/// Filter by namespace
#[arg(short, long)]
namespace: Option<String>,
/// Output as JSON
#[arg(long)]
json: bool,
},
/// List TLS certificates (from Secrets)
Certs {
/// Filter by namespace
#[arg(short, long)]
namespace: Option<String>,
/// Output as JSON
#[arg(long)]
json: bool,
},
/// Validate Ingress configurations
Validate {
/// Filter by namespace
#[arg(short, long)]
namespace: Option<String>,
},
/// Show GIngress controller status and summary
Status {
/// Output as JSON
#[arg(long)]
json: bool,
},
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
let cli = Cli::parse();
let client = Client::try_default().await?;
match cli.command {
Command::List { namespace, json } => cmd_list(&client, namespace, json).await?,
Command::Routes {
namespace,
host,
json,
} => cmd_routes(&client, namespace, host, json).await?,
Command::Backends { namespace, json } => cmd_backends(&client, namespace, json).await?,
Command::Certs { namespace, json } => cmd_certs(&client, namespace, json).await?,
Command::Validate { namespace } => cmd_validate(&client, namespace).await?,
Command::Status { json } => cmd_status(&client, json).await?,
}
Ok(())
}
// ── list ──────────────────────────────────────────────────────────
async fn cmd_list(
client: &Client,
namespace: Option<String>,
json: bool,
) -> Result<(), Box<dyn std::error::Error>> {
let ingresses = list_ingresses(client, namespace.as_deref()).await?;
if json {
println!("{}", serde_json::to_string_pretty(&ingresses)?);
return Ok(());
}
if ingresses.is_empty() {
println!("No GIngress-managed Ingress resources found.");
return Ok(());
}
println!(
"{:<25} {:<20} {:<40} {:<50} {:<15}",
"NAMESPACE", "NAME", "HOSTS", "PATHS", "TLS"
);
println!("{:-<150}", "");
for ing in &ingresses {
let ns = ing.namespace();
let name = ing.name_any();
let hosts = ing.hosts().join(", ");
let paths = ing
.paths_display()
.iter()
.map(|p| format!("{} {}", p.path_type, p.path))
.collect::<Vec<_>>()
.join(", ");
let tls = if ing.has_tls() { "Enabled" } else { "-" };
println!(
"{:<25} {:<20} {:<40} {:<50} {:<15}",
truncate(&ns, 25),
truncate(&name, 20),
truncate(&hosts, 40),
truncate(&paths, 50),
tls,
);
}
println!("\nTotal: {} Ingress(es)", ingresses.len());
Ok(())
}
// ── routes ─────────────────────────────────────────────────────────
async fn cmd_routes(
client: &Client,
namespace: Option<String>,
host_filter: Option<String>,
json: bool,
) -> Result<(), Box<dyn std::error::Error>> {
let ingresses = list_ingresses(client, namespace.as_deref()).await?;
let mut routes: Vec<RouteRow> = Vec::new();
for ing in &ingresses {
for rule in ing
.spec
.as_ref()
.and_then(|s| s.rules.as_ref())
.into_iter()
.flatten()
{
let host = rule.host.as_deref().unwrap_or("*");
if let Some(ref hf) = host_filter {
if host != hf {
continue;
}
}
if let Some(http) = &rule.http {
for path_item in &http.paths {
let backend = extract_backend(path_item);
let port = extract_backend_port(path_item);
routes.push(RouteRow {
namespace: ing.namespace(),
ingress: ing.name_any(),
host: host.to_string(),
path: path_item.path.clone().unwrap_or_else(|| "/".into()),
path_type: path_item.path_type.clone(),
backend,
port,
});
}
}
}
}
if json {
println!("{}", serde_json::to_string_pretty(&routes)?);
return Ok(());
}
if routes.is_empty() {
println!("No routes found.");
return Ok(());
}
println!(
"{:<20} {:<20} {:<30} {:<18} {:<15} {:<15} {:<15}",
"NAMESPACE", "INGRESS", "HOST", "PATH", "TYPE", "BACKEND", "PORT"
);
println!("{:-<133}", "");
for r in &routes {
let port = extract_backend_port_str(r);
println!(
"{:<20} {:<20} {:<30} {:<18} {:<15} {:<15} {:<15}",
truncate(&r.namespace, 20),
truncate(&r.ingress, 20),
truncate(&r.host, 30),
truncate(&r.path, 18),
truncate(&r.path_type, 15),
truncate(&r.backend, 15),
port,
);
}
println!("\nTotal: {} route(s)", routes.len());
Ok(())
}
// ── backends ───────────────────────────────────────────────────────
async fn cmd_backends(
client: &Client,
namespace: Option<String>,
json: bool,
) -> Result<(), Box<dyn std::error::Error>> {
let ingresses = list_ingresses(client, namespace.as_deref()).await?;
// Collect unique backends from all ingresses
let mut backends: Vec<BackendRow> = Vec::new();
let mut seen = std::collections::HashSet::new();
for ing in &ingresses {
for rule in ing
.spec
.as_ref()
.and_then(|s| s.rules.as_ref())
.into_iter()
.flatten()
{
if let Some(http) = &rule.http {
for path_item in &http.paths {
let svc = match path_item.backend.service.as_ref() {
Some(s) => s,
None => continue,
};
let key = format!(
"{}/{}:{}",
ing.namespace(),
svc.name,
svc.port.as_ref().and_then(|p| p.number).unwrap_or(80)
);
if seen.insert(key.clone()) {
let ns = ing.namespace();
let ep_status = get_endpoint_status(client, &ns, &svc.name).await;
backends.push(BackendRow {
namespace: ns,
service: svc.name.clone(),
port: svc.port.as_ref().and_then(|p| p.number).unwrap_or(80) as u16,
ready_endpoints: ep_status.ready,
total_endpoints: ep_status.total,
referenced_by: ing.name_any(),
});
}
}
}
}
}
if json {
println!("{}", serde_json::to_string_pretty(&backends)?);
return Ok(());
}
if backends.is_empty() {
println!("No backends found.");
return Ok(());
}
println!(
"{:<20} {:<20} {:<8} {:<8} {:<18} {:<20}",
"NAMESPACE", "SERVICE", "PORT", "HEALTH", "ENDPOINTS", "REFERENCED BY"
);
println!("{:-<94}", "");
for b in &backends {
let health = if b.total_endpoints == 0 {
"WARN"
} else if b.ready_endpoints == 0 {
"DOWN"
} else if b.ready_endpoints < b.total_endpoints {
"PARTIAL"
} else {
"OK"
};
let eps = format!("{}/{} ready", b.ready_endpoints, b.total_endpoints);
println!(
"{:<20} {:<20} {:<8} {:<8} {:<18} {:<20}",
truncate(&b.namespace, 20),
truncate(&b.service, 20),
b.port,
health,
eps,
truncate(&b.referenced_by, 20),
);
}
println!("\nTotal: {} backend(s)", backends.len());
Ok(())
}
// ── certs ──────────────────────────────────────────────────────────
async fn cmd_certs(
client: &Client,
namespace: Option<String>,
json: bool,
) -> Result<(), Box<dyn std::error::Error>> {
let ingresses = list_ingresses(client, namespace.as_deref()).await?;
let mut certs: Vec<CertRow> = Vec::new();
for ing in &ingresses {
let ns = ing.namespace();
let tls_entries = ing
.spec
.as_ref()
.and_then(|s| s.tls.as_ref())
.cloned()
.unwrap_or_default();
for tls in &tls_entries {
let secret_name = tls.secret_name.clone().unwrap_or_default();
let hosts = tls.hosts.clone().unwrap_or_default();
// Check if the secret exists
let secret_exists = check_secret_exists(client, &ns, &secret_name).await;
for host in &hosts {
certs.push(CertRow {
namespace: ns.clone(),
secret_name: secret_name.clone(),
host: host.clone(),
found: secret_exists,
});
}
}
}
if json {
println!("{}", serde_json::to_string_pretty(&certs)?);
return Ok(());
}
if certs.is_empty() {
println!("No TLS certificates configured.");
return Ok(());
}
println!(
"{:<20} {:<30} {:<30} {:<10}",
"NAMESPACE", "SECRET", "HOST", "STATUS"
);
println!("{:-<90}", "");
for c in &certs {
let status = if c.found { "OK" } else { "MISSING" };
println!(
"{:<20} {:<30} {:<30} {:<10}",
truncate(&c.namespace, 20),
truncate(&c.secret_name, 30),
truncate(&c.host, 30),
status,
);
}
let missing = certs.iter().filter(|c| !c.found).count();
println!("\nTotal: {} cert(s), {} missing", certs.len(), missing);
Ok(())
}
// ── validate ───────────────────────────────────────────────────────
async fn cmd_validate(
client: &Client,
namespace: Option<String>,
) -> Result<(), Box<dyn std::error::Error>> {
let ingresses = list_ingresses(client, namespace.as_deref()).await?;
let mut errors = 0usize;
let mut warnings = 0usize;
for ing in &ingresses {
let ns = ing.namespace();
let name = ing.name_any();
// Check: has rules
let has_rules = ing
.spec
.as_ref()
.map(|s| s.rules.as_ref().map(|r| !r.is_empty()).unwrap_or(false))
.unwrap_or(false);
if !has_rules {
println!("[{}/{}] ERROR: No routing rules defined", ns, name);
errors += 1;
}
// Check: has TLS but no secret
let tls_entries = ing
.spec
.as_ref()
.and_then(|s| s.tls.as_ref())
.cloned()
.unwrap_or_default();
for tls in &tls_entries {
let secret_name = tls.secret_name.as_deref().unwrap_or("");
if secret_name.is_empty() {
println!(
"[{}/{}] WARNING: TLS configured but no secretName specified",
ns, name
);
warnings += 1;
} else {
let found = check_secret_exists(client, &ns, secret_name).await;
if !found {
println!(
"[{}/{}] ERROR: TLS secret '{}' not found in namespace '{}'",
ns, name, secret_name, ns
);
errors += 1;
}
}
}
// Check: path backends reference valid services
if let Some(rules) = ing.spec.as_ref().and_then(|s| s.rules.as_ref()) {
for rule in rules {
if let Some(http) = &rule.http {
for path_item in &http.paths {
if let Some(svc) = &path_item.backend.service {
let endpoints = get_endpoint_status(client, &ns, &svc.name).await;
if endpoints.total == 0 {
println!(
"[{}/{}] WARNING: Backend service '{}' has no endpoints (host: {})",
ns,
name,
svc.name,
rule.host.as_deref().unwrap_or("*")
);
warnings += 1;
}
}
}
}
}
}
}
if errors == 0 && warnings == 0 {
println!(
"Validation passed — no issues found in {} Ingress(es).",
ingresses.len()
);
} else {
println!(
"\nValidation complete: {} error(s), {} warning(s) across {} Ingress(es).",
errors,
warnings,
ingresses.len()
);
}
Ok(())
}
// ── status ─────────────────────────────────────────────────────────
async fn cmd_status(client: &Client, json: bool) -> Result<(), Box<dyn std::error::Error>> {
let ingresses = list_ingresses(client, None).await?;
let controller_pods = find_gingress_pods(client).await;
if json {
#[derive(serde::Serialize)]
struct StatusOutput {
controller_pods: Vec<PodInfo>,
ingress_count: usize,
route_count: usize,
backend_count: usize,
cert_count: usize,
}
let mut route_count = 0usize;
let mut backend_set = std::collections::HashSet::new();
let mut cert_count = 0usize;
for ing in &ingresses {
if let Some(spec) = &ing.spec {
if let Some(rules) = &spec.rules {
for rule in rules {
if let Some(http) = &rule.http {
route_count += http.paths.len();
for p in &http.paths {
if let Some(svc) = &p.backend.service {
backend_set.insert(format!("{}/{}", ing.namespace(), svc.name));
}
}
}
}
}
if let Some(tls) = &spec.tls {
cert_count += tls.len();
}
}
}
println!(
"{}",
serde_json::to_string_pretty(&StatusOutput {
controller_pods,
ingress_count: ingresses.len(),
route_count,
backend_count: backend_set.len(),
cert_count,
})?
);
return Ok(());
}
// Controller status
println!("══ GIngress Controller Status ══\n");
if controller_pods.is_empty() {
println!(
"Controller: NOT FOUND (no pods with label gingress.io/component=controller)"
);
} else {
for pod in &controller_pods {
let ready = if pod.ready { "Running" } else { "NotReady" };
println!(
"Controller Pod: {:<30} {:<12} {}",
truncate(&pod.name, 30),
ready,
pod.namespace
);
}
}
// Resource summary
println!();
println!("══ Managed Resources ══\n");
let mut route_count = 0usize;
let mut backend_set = std::collections::HashSet::new();
let mut backend_total_eps = 0usize;
let mut backend_ready_eps = 0usize;
let mut cert_count = 0usize;
let mut cert_missing = 0usize;
for ing in &ingresses {
if let Some(spec) = &ing.spec {
if let Some(rules) = &spec.rules {
for rule in rules {
if let Some(http) = &rule.http {
route_count += http.paths.len();
for p in &http.paths {
if let Some(svc) = &p.backend.service {
let key = format!("{}/{}", ing.namespace(), svc.name);
if backend_set.insert(key) {
let eps =
get_endpoint_status(client, &ing.namespace(), &svc.name)
.await;
backend_total_eps += eps.total;
backend_ready_eps += eps.ready;
}
}
}
}
}
}
if let Some(tls) = &spec.tls {
cert_count += tls.len();
for tls in tls {
let sn = tls.secret_name.as_deref().unwrap_or("");
if !sn.is_empty() && !check_secret_exists(client, &ing.namespace(), sn).await {
cert_missing += 1;
}
}
}
}
}
println!("Ingresses: {}", ingresses.len());
println!("Routes: {}", route_count);
println!(
"Backends: {} ({} ready / {} total endpoints)",
backend_set.len(),
backend_ready_eps,
backend_total_eps
);
println!(
"TLS Certs: {} ({} missing)",
cert_count, cert_missing
);
println!();
// Overall health
let healthy = !ingresses.is_empty()
&& (cert_missing == 0)
&& (backend_set.is_empty() || backend_ready_eps > 0)
&& controller_pods.iter().any(|p| p.ready);
if healthy {
println!("Status: HEALTHY");
} else {
println!("Status: DEGRADED");
if controller_pods.is_empty() || !controller_pods.iter().any(|p| p.ready) {
println!(" → Controller pod not running or not ready");
}
if cert_missing > 0 {
println!("{} TLS secret(s) missing", cert_missing);
}
if !backend_set.is_empty() && backend_ready_eps == 0 {
println!(" → No ready backend endpoints");
}
}
Ok(())
}
// ── k8s helpers ────────────────────────────────────────────────────
#[derive(serde::Serialize)]
struct IngressSummary {
namespace: String,
name: String,
hosts: Vec<String>,
#[serde(skip)]
paths_for_display: Vec<PathSummary>,
has_tls: bool,
#[serde(skip)]
spec: Option<k8s_openapi::api::networking::v1::IngressSpec>,
}
#[derive(Clone)]
struct PathSummary {
path_type: String,
path: String,
}
impl IngressSummary {
fn namespace(&self) -> String {
self.namespace.clone()
}
fn name_any(&self) -> String {
self.name.clone()
}
fn hosts(&self) -> &[String] {
&self.hosts
}
fn paths_display(&self) -> &[PathSummary] {
&self.paths_for_display
}
fn has_tls(&self) -> bool {
self.has_tls
}
}
fn ingress_to_summary(ing: &Ingress) -> IngressSummary {
let spec = ing.spec.clone();
let mut hosts = Vec::new();
let mut paths = Vec::new();
let mut has_tls = false;
if let Some(ref s) = spec {
if let Some(ref rules) = s.rules {
for rule in rules {
if let Some(ref host) = rule.host {
hosts.push(host.clone());
}
if let Some(ref http) = rule.http {
for p in &http.paths {
paths.push(PathSummary {
path_type: p.path_type.clone(),
path: p.path.clone().unwrap_or_else(|| "/".into()),
});
}
}
}
}
has_tls = s.tls.as_ref().map(|t| !t.is_empty()).unwrap_or(false);
}
IngressSummary {
namespace: ing.namespace().unwrap_or_default(),
name: ing.name_any(),
hosts,
paths_for_display: paths,
has_tls,
spec,
}
}
#[derive(serde::Serialize)]
struct PodInfo {
name: String,
namespace: String,
ready: bool,
}
/// Find GIngress controller pods by label `gingress.io/component=controller`.
async fn find_gingress_pods(client: &Client) -> Vec<PodInfo> {
let api: Api<Pod> = Api::all(client.clone());
let lp = ListParams {
label_selector: Some("gingress.io/component=controller".into()),
..Default::default()
};
match api.list(&lp).await {
Ok(list) => list
.items
.into_iter()
.map(|pod| {
let ready = pod
.status
.as_ref()
.and_then(|s| s.conditions.as_ref())
.map(|conds| {
conds
.iter()
.any(|c| c.type_ == "Ready" && c.status == "True")
})
.unwrap_or(false);
PodInfo {
name: pod.name_any(),
namespace: pod.namespace().unwrap_or_default(),
ready,
}
})
.collect(),
Err(_) => Vec::new(),
}
}
async fn list_ingresses(
client: &Client,
namespace: Option<&str>,
) -> Result<Vec<IngressSummary>, Box<dyn std::error::Error>> {
let params = ListParams {
..Default::default()
};
if let Some(ns) = namespace {
let api: Api<Ingress> = Api::namespaced(client.clone(), ns);
let list = api.list(&params).await?;
Ok(list
.items
.into_iter()
.filter(|ing| is_gingress_class(ing))
.map(|ing| ingress_to_summary(&ing))
.collect())
} else {
let api: Api<Ingress> = Api::all(client.clone());
let list = api.list(&params).await?;
Ok(list
.items
.into_iter()
.filter(|ing| is_gingress_class(ing))
.map(|ing| ingress_to_summary(&ing))
.collect())
}
}
fn is_gingress_class(ingress: &Ingress) -> bool {
ingress
.spec
.as_ref()
.and_then(|s| s.ingress_class_name.as_deref())
== Some(INGRESS_CLASS)
}
struct EndpointStatus {
ready: usize,
total: usize,
}
async fn get_endpoint_status(
client: &Client,
namespace: &str,
service_name: &str,
) -> EndpointStatus {
use k8s_openapi::api::core::v1::Endpoints;
let api: Api<Endpoints> = Api::namespaced(client.clone(), namespace);
match api.get_opt(service_name).await {
Ok(Some(eps)) => {
let mut ready = 0usize;
let mut total = 0usize;
if let Some(subsets) = &eps.subsets {
for subset in subsets {
let addrs = subset.addresses.as_deref().unwrap_or_default();
let not_ready = subset.not_ready_addresses.as_deref().unwrap_or_default();
ready += addrs.len();
total += addrs.len() + not_ready.len();
}
}
EndpointStatus { ready, total }
}
_ => EndpointStatus { ready: 0, total: 0 },
}
}
async fn check_secret_exists(client: &Client, namespace: &str, name: &str) -> bool {
let api: Api<Secret> = Api::namespaced(client.clone(), namespace);
api.get_opt(name).await.ok().flatten().is_some()
}
fn extract_backend(path_item: &HTTPIngressPath) -> String {
path_item
.backend
.service
.as_ref()
.map(|s| s.name.clone())
.unwrap_or_else(|| "<resource>".into())
}
fn extract_backend_port(path_item: &HTTPIngressPath) -> u16 {
path_item
.backend
.service
.as_ref()
.and_then(|s| s.port.as_ref())
.and_then(|p| p.number)
.unwrap_or(80) as u16
}
fn extract_backend_port_str(r: &RouteRow) -> String {
r.port.to_string()
}
#[derive(serde::Serialize)]
struct RouteRow {
namespace: String,
ingress: String,
host: String,
path: String,
path_type: String,
backend: String,
port: u16,
}
#[derive(serde::Serialize)]
struct BackendRow {
namespace: String,
service: String,
port: u16,
ready_endpoints: usize,
total_endpoints: usize,
referenced_by: String,
}
#[derive(serde::Serialize)]
struct CertRow {
namespace: String,
secret_name: String,
host: String,
found: bool,
}
fn truncate(s: &str, max: usize) -> String {
// Account for CJK characters — each wide char counts as 2
let mut width = 0usize;
let mut result = String::new();
for c in s.chars() {
let cw = if c.is_ascii() { 1 } else { 2 };
if width + cw > max {
result.push_str("");
break;
}
result.push(c);
width += cw;
}
result
}

View File

@ -1,148 +0,0 @@
//! Watches Kubernetes Endpoints and updates upstream endpoint lists.
//!
//! Tracks Pod IPs for each Service. When endpoints change (scale up/down,
//! rolling restart, health check failures), the upstream pool is updated.
use futures::StreamExt;
use futures::pin_mut;
use gingress_proxy::config::{ConfigStore, Endpoint};
use k8s_openapi::api::core::v1::Endpoints as K8sEndpoints;
use kube::ResourceExt;
use kube::runtime::watcher::{self, Event};
use std::sync::Arc;
/// Watch Endpoints and update the ConfigStore.
pub async fn watch_endpoints(
client: Arc<kube::Client>,
store: Arc<ConfigStore>,
_namespace: Option<String>,
on_change: Arc<dyn Fn() + Send + Sync>,
) {
let api = kube::Api::<K8sEndpoints>::all(client.as_ref().clone());
let config = watcher::Config::default();
let watcher = watcher::watcher(api, config);
pin_mut!(watcher);
while let Some(event) = watcher.next().await {
match event {
Ok(Event::Apply(eps)) => {
process_endpoints(&eps, &store, &on_change);
}
Ok(Event::Init) => {
tracing::info!("Endpoint watcher re-initializing");
}
Ok(Event::InitApply(eps)) => {
process_endpoints(&eps, &store, &on_change);
}
Ok(Event::InitDone) => {
tracing::info!("Endpoint watcher init complete");
}
Ok(Event::Delete(eps)) => {
remove_endpoints(&eps, &store, &on_change);
}
Err(e) => {
tracing::error!("Endpoint watcher error: {}", e);
}
}
}
}
/// Extract endpoint addresses, grouped by port, and update the ConfigStore.
///
/// Stores endpoints under key `upstream:<ns>/<name>:<port>` to match
/// the proxy's upstream lookup format.
fn process_endpoints(
endpoints: &K8sEndpoints,
store: &ConfigStore,
on_change: &Arc<dyn Fn() + Send + Sync>,
) {
use std::collections::HashMap;
let name = endpoints.name_any();
let namespace = endpoints.namespace().unwrap_or_default();
let base_prefix = format!("upstream:{}/{}:", namespace, name);
// Collect endpoints grouped by port
let mut port_groups: HashMap<u16, Vec<Endpoint>> = HashMap::new();
if let Some(subsets) = &endpoints.subsets {
for subset in subsets {
let addrs = subset.addresses.as_deref().unwrap_or_default();
let ports = subset.ports.as_deref().unwrap_or_default();
let not_ready_addrs = subset.not_ready_addresses.as_deref().unwrap_or_default();
for port in ports {
let port_num = port.port as u16;
let eps = port_groups.entry(port_num).or_default();
for addr in addrs {
eps.push(Endpoint {
ip: addr.ip.clone(),
port: port_num,
ready: true,
});
}
for addr in not_ready_addrs {
eps.push(Endpoint {
ip: addr.ip.clone(),
port: port_num,
ready: false,
});
}
}
}
}
// Clear old per-port keys for this service (handles port removal)
let old_keys = store.keys_with_prefix(&base_prefix);
for k in old_keys {
store.remove(&k);
}
// Write per-port endpoint entries
let mut total = 0usize;
for (port_num, eps) in &port_groups {
let key = format!("{}{}", base_prefix, port_num);
store.set(&key, eps);
total += eps.len();
}
// If no ports at all, write an empty entry for the base key so the reconciler
// can detect that this service has no endpoints.
if port_groups.is_empty() {
store.set::<Vec<Endpoint>>(&format!("upstream:{}/{}", namespace, name), &vec![]);
}
store.signal_reload();
on_change();
tracing::debug!(
namespace = %namespace,
name = %name,
num_ports = port_groups.len(),
num_endpoints = total,
"Endpoints updated"
);
}
/// Remove all per-port endpoint keys when the Endpoint resource is deleted.
fn remove_endpoints(
endpoints: &K8sEndpoints,
store: &ConfigStore,
on_change: &Arc<dyn Fn() + Send + Sync>,
) {
let name = endpoints.name_any();
let namespace = endpoints.namespace().unwrap_or_default();
let base_prefix = format!("upstream:{}/{}:", namespace, name);
// Remove all per-port keys
let keys = store.keys_with_prefix(&base_prefix);
for k in keys {
store.remove(&k);
}
// Also remove the port-less key (in case no ports were present)
store.remove(&format!("upstream:{}/{}", namespace, name));
store.signal_reload();
on_change();
tracing::info!(namespace = %namespace, name = %name, "Endpoints removed");
}

View File

@ -1,426 +0,0 @@
//! Watches Kubernetes Ingress resources and converts them to routing rules.
use futures::StreamExt;
use futures::pin_mut;
use gingress_proxy::config::{
ConfigStore, HeaderOp, PathType, RateLimitPolicy, RouteRule, SessionAffinityConfig,
};
use k8s_openapi::api::networking::v1::{HTTPIngressPath, Ingress};
use kube::ResourceExt;
use kube::runtime::watcher::{self, Event};
use std::collections::BTreeMap;
use std::sync::Arc;
/// Watch Ingress resources and update the ConfigStore.
///
/// After each event, the `on_change` callback is invoked so the reconciler
/// can cross-reference all fragments into a complete ProxyConfig.
pub async fn watch_ingresses(
client: Arc<kube::Client>,
store: Arc<ConfigStore>,
ingress_class: String,
namespace: Option<String>,
on_change: Arc<dyn Fn() + Send + Sync>,
) {
let api = kube::Api::<Ingress>::all(client.as_ref().clone());
let config = watcher::Config {
field_selector: namespace
.as_ref()
.map(|ns| format!("metadata.namespace={}", ns)),
..Default::default()
};
let ingress_watcher = watcher::watcher(api, config);
pin_mut!(ingress_watcher);
while let Some(event) = ingress_watcher.next().await {
match event {
Ok(Event::Apply(ingress)) => {
let name = ingress.name_any();
let ns = ingress.namespace().unwrap_or_default();
if is_gingress_class(&ingress, &ingress_class) {
process_ingress(&ingress, &store, &ingress_class);
on_change();
tracing::info!(namespace = %ns, name = %name, "Ingress applied");
}
}
Ok(Event::Init) => {
store.remove_prefix("ingress:");
store.remove_prefix("tls-host:");
tracing::info!("Ingress watcher re-initializing");
}
Ok(Event::InitApply(ingress)) => {
if is_gingress_class(&ingress, &ingress_class) {
process_ingress(&ingress, &store, &ingress_class);
}
}
Ok(Event::InitDone) => {
store.signal_reload();
on_change();
tracing::info!("Ingress watcher init complete");
}
Ok(Event::Delete(ingress)) => {
if is_gingress_class(&ingress, &ingress_class) {
remove_ingress_routes(&ingress, &store);
on_change();
tracing::info!(
name = %ingress.name_any(),
namespace = %ingress.namespace().unwrap_or_default(),
"Ingress deleted, routes removed"
);
}
}
Err(e) => {
tracing::error!("Ingress watcher error: {}", e);
}
}
}
}
/// Check if an Ingress specifies the gingress class.
fn is_gingress_class(ingress: &Ingress, class_name: &str) -> bool {
ingress
.spec
.as_ref()
.and_then(|s| s.ingress_class_name.as_deref())
== Some(class_name)
}
/// Process an Ingress resource: extract routes and update the store.
fn process_ingress(ingress: &Ingress, store: &ConfigStore, _ingress_class: &str) {
let namespace = ingress.namespace().unwrap_or_default();
let name = ingress.name_any();
let spec = match ingress.spec.as_ref() {
Some(s) => s,
None => return,
};
// Build an ingress-scoped prefix so we can clean up old routes for this Ingress
let ingress_prefix = format!("ingress:{}/{}:", namespace, name);
// Remove old route entries scoped to this Ingress
let old_route_keys = store.keys_with_prefix(&format!("{}route:", ingress_prefix));
for key in &old_route_keys {
store.remove(key);
}
// Process routing rules
if let Some(rules) = &spec.rules {
for rule in rules {
let host = rule.host.as_deref().unwrap_or("*");
if let Some(http) = &rule.http {
let mut routes: Vec<RouteRule> = Vec::new();
for path_item in &http.paths {
routes.push(ingress_path_to_route(host, path_item, &namespace));
}
// Store per-ingress routes so we can clean up on delete
let route_key = format!("{}route:{}", ingress_prefix, host);
store.set(&route_key, &routes);
}
}
}
// Process TLS: map secretName -> hosts so the reconciler can cross-reference
if let Some(tls_entries) = &spec.tls {
for tls in tls_entries {
let secret_name = tls.secret_name.as_deref().unwrap_or_default();
let hosts: Vec<String> = tls.hosts.clone().unwrap_or_default();
let tls_host_key = format!("tls-host:{}", secret_name);
store.set(&tls_host_key, &hosts);
}
}
// Process annotations for advanced features
let annotations = ingress.annotations();
process_annotations(&annotations, &ingress_prefix, &namespace, store);
store.signal_reload();
}
/// Convert a Kubernetes Ingress path to an internal RouteRule.
fn ingress_path_to_route(host: &str, path: &HTTPIngressPath, namespace: &str) -> RouteRule {
let service = path
.backend
.service
.as_ref()
.expect("Ingress backend must reference a service");
RouteRule {
host: host.to_string(),
path: path.path.clone().unwrap_or_else(|| "/".to_string()),
path_type: match path.path_type.as_str() {
"Prefix" => PathType::Prefix,
"Exact" => PathType::Exact,
_ => PathType::ImplementationSpecific,
},
backend: gingress_proxy::config::Backend {
namespace: namespace.to_string(),
name: service.name.clone(),
port: service.port.as_ref().and_then(|p| p.number).unwrap_or(80) as u16,
},
}
}
/// Annotation keys for GIngress features.
const ANN_RATE_LIMIT: &str = "gingress.io/rate-limit";
const ANN_RATE_LIMIT_BURST: &str = "gingress.io/rate-limit-burst";
const ANN_REQUEST_HEADERS: &str = "gingress.io/request-headers";
const ANN_WEBSOCKET: &str = "gingress.io/websocket";
const ANN_SESSION_AFFINITY: &str = "gingress.io/session-affinity";
const ANN_GIT_BACKEND: &str = "gingress.io/git-backend";
/// Parse Ingress annotations and write corresponding ConfigStore entries.
///
/// Supported annotations:
/// - `gingress.io/rate-limit` — "RPS" or "RPS/BURST" (e.g., "100" or "100/200")
/// - `gingress.io/rate-limit-burst` — Override burst size
/// - `gingress.io/request-headers` — JSON array of header operations
/// - `gingress.io/websocket` — "true" to enable WebSocket upgrade for this host
/// - `gingress.io/session-affinity` — "cookie" or "cookie:NAME:TTL_SECONDS"
fn process_annotations(
annotations: &BTreeMap<String, String>,
ingress_prefix: &str,
namespace: &str,
store: &ConfigStore,
) {
// Collect hosts from the ingress routes that were just stored
let route_keys = store.keys_with_prefix(&format!("{}route:", ingress_prefix));
let hosts: Vec<String> = route_keys
.iter()
.filter_map(|k| k.split(":route:").nth(1).map(String::from))
.collect();
if hosts.is_empty() {
return;
}
// Remove old per-host annotation keys (handles annotation removal/update)
for host in &hosts {
store.remove(&format!("rate_limit:{}", host));
store.remove(&format!("headers:{}", host));
store.remove(&format!("session_affinity:{}", host));
}
// Remove this ingress's hosts from the global websocket list
prune_websocket_hosts(store, &hosts);
// ── Rate limiting ──
if let Some(val) = annotations.get(ANN_RATE_LIMIT) {
let (rps, burst) = parse_rate_limit(val, annotations.get(ANN_RATE_LIMIT_BURST));
for host in &hosts {
store.set(
&format!("rate_limit:{}", host),
&RateLimitPolicy {
host: host.clone(),
requests_per_second: rps,
burst_size: burst,
},
);
}
}
// ── Header operations (request) ──
if let Some(val) = annotations.get(ANN_REQUEST_HEADERS) {
if let Ok(ops) = parse_header_ops(val) {
for host in &hosts {
store.set(&format!("headers:{}", host), &ops);
}
} else {
tracing::warn!(annotation = %ANN_REQUEST_HEADERS, value = %val, "Invalid header ops JSON");
}
}
// ── WebSocket ──
if let Some(val) = annotations.get(ANN_WEBSOCKET) {
if val.trim().to_lowercase() == "true" {
let mut ws_hosts: Vec<String> = hosts.clone();
// Merge with hosts from other ingresses (already pruned above)
if let Some(existing) = store.get::<Vec<String>>("websocket:hosts") {
for h in existing {
if !ws_hosts.contains(&h) {
ws_hosts.push(h);
}
}
}
store.set("websocket:hosts", &ws_hosts);
}
}
// ── Session affinity ──
if let Some(val) = annotations.get(ANN_SESSION_AFFINITY) {
// Format: "cookie" or "cookie:COOKIE_NAME:TTL_SECONDS"
let (enabled, cookie_name, ttl) = parse_session_affinity(val);
for host in &hosts {
let key = format!("session_affinity:{}", host);
store.set(
&key,
&SessionAffinityConfig {
enabled,
cookie_name: cookie_name.clone(),
cookie_ttl_seconds: ttl,
},
);
}
}
// ── Git backend ──
// When present, requests with Git User-Agent (git/*, JGit/*) are routed to
// this backend instead of normal host+path matching.
// Value format: "namespace/service-name:port" or "service-name:port" (namespace from Ingress)
if let Some(val) = annotations.get(ANN_GIT_BACKEND) {
if let Some(backend) = parse_git_backend(val, &namespace) {
store.set("git_backend", &backend);
tracing::info!(
backend = format!("{}/{}:{}", backend.namespace, backend.name, backend.port),
"Git backend configured"
);
} else {
tracing::warn!(annotation = %ANN_GIT_BACKEND, value = %val, "Invalid git-backend format, expected 'namespace/name:port' or 'name:port'");
}
}
}
fn parse_rate_limit(val: &str, burst_override: Option<&String>) -> (u32, u32) {
let val = val.trim();
if let Some((rps_str, burst_str)) = val.split_once('/') {
let rps = rps_str.parse().unwrap_or(0);
let burst = burst_str.parse().unwrap_or(rps);
(rps, burst)
} else {
let rps = val.parse().unwrap_or(0);
let burst = burst_override
.and_then(|b| b.parse().ok())
.unwrap_or(rps * 2);
(rps, burst)
}
}
#[derive(serde::Deserialize)]
struct HeaderOpAnnotation {
op: String,
name: String,
#[serde(default)]
value: Option<String>,
}
fn parse_header_ops(val: &str) -> anyhow::Result<Vec<HeaderOp>> {
let items: Vec<HeaderOpAnnotation> = serde_json::from_str(val)?;
items
.into_iter()
.map(|item| {
Ok(match item.op.as_str() {
"set" => HeaderOp::Set {
name: item.name,
value: item.value.unwrap_or_default(),
},
"add" => HeaderOp::Add {
name: item.name,
value: item.value.unwrap_or_default(),
},
"remove" => HeaderOp::Remove { name: item.name },
_ => anyhow::bail!("Unknown header op: {}", item.op),
})
})
.collect()
}
fn parse_session_affinity(val: &str) -> (bool, String, u64) {
let val = val.trim();
if val.eq_ignore_ascii_case("cookie") || val.eq_ignore_ascii_case("true") {
return (true, "GINGRESS_AFFINITY".into(), 3600);
}
// Format: "cookie:COOKIE_NAME:TTL"
let parts: Vec<&str> = val.split(':').collect();
if parts.len() >= 3 {
let name = parts[1].to_string();
let ttl = parts[2].parse().unwrap_or(3600);
(true, name, ttl)
} else if parts.len() == 2 {
let name = parts[1].to_string();
(true, name, 3600)
} else {
(false, String::new(), 0)
}
}
/// Parse git-backend annotation value.
///
/// Format: "namespace/name:port" or "name:port" (namespace defaults to Ingress namespace).
fn parse_git_backend(
val: &str,
default_namespace: &str,
) -> Option<gingress_proxy::config::Backend> {
let val = val.trim();
// Split off port: "namespace/name:port" → ("namespace/name", "port")
let (ns_name, port_str) = val.rsplit_once(':').unwrap_or((val, ""));
let port: u16 = port_str.parse().ok()?;
// Split namespace and name: "namespace/name" → ("namespace", "name")
let (namespace, name) = if let Some((ns, n)) = ns_name.rsplit_once('/') {
(ns.to_string(), n.to_string())
} else {
// No namespace specified — use the Ingress namespace
(default_namespace.to_string(), ns_name.to_string())
};
if name.is_empty() {
return None;
}
Some(gingress_proxy::config::Backend {
namespace,
name,
port,
})
}
/// Remove a set of hosts from the global websocket host list (scoped cleanup).
fn prune_websocket_hosts(store: &ConfigStore, hosts_to_remove: &[String]) {
if let Some(mut existing) = store.get::<Vec<String>>("websocket:hosts") {
existing.retain(|h| !hosts_to_remove.contains(h));
if existing.is_empty() {
store.remove("websocket:hosts");
} else {
store.set("websocket:hosts", &existing);
}
}
}
/// Remove all routes associated with a deleted Ingress.
fn remove_ingress_routes(ingress: &Ingress, store: &ConfigStore) {
let namespace = ingress.namespace().unwrap_or_default();
let name = ingress.name_any();
let ingress_prefix = format!("ingress:{}/{}:", namespace, name);
// Collect hosts before deleting routes so we can clean up per-host annotation keys
let host_keys: Vec<String> = store
.keys_with_prefix(&format!("{}route:", ingress_prefix))
.iter()
.filter_map(|k| k.split(":route:").nth(1).map(String::from))
.collect();
// Remove all route entries for this Ingress
store.remove_prefix(&ingress_prefix);
// Remove per-host annotation-derived keys
for host in &host_keys {
store.remove(&format!("rate_limit:{}", host));
store.remove(&format!("headers:{}", host));
store.remove(&format!("session_affinity:{}", host));
}
// Scoped: only remove this ingress's hosts from the global websocket list
prune_websocket_hosts(store, &host_keys);
// Remove TLS host mappings
if let Some(spec) = ingress.spec.as_ref() {
if let Some(tls_entries) = &spec.tls {
for tls in tls_entries {
let sn = tls.secret_name.as_deref().unwrap_or_default();
store.remove(&format!("tls-host:{}", sn));
}
}
}
store.signal_reload();
}

View File

@ -1,88 +0,0 @@
//! Kubernetes controller for GIngress.
//!
//! Watches Ingress, Service, EndpointSlice, and Secret resources,
//! reconciles them into the shared `ConfigStore`.
mod endpoint_watcher;
mod ingress_watcher;
mod reconciler;
mod secret_watcher;
use anyhow::Context;
use gingress_proxy::config::ConfigStore;
use kube::Client;
use std::sync::Arc;
use tokio::task::JoinHandle;
/// Start all controller watchers and the reconcile loop.
///
/// Each watcher:
/// 1. Watches a specific K8s resource type
/// 2. Writes its fragment to the `ConfigStore`
/// 3. Calls `reconciler.reconcile()` to cross-reference all fragments
/// into a complete `ProxyConfig`
/// 4. The reconiler signals `ConfigStore::signal_reload()`
/// 5. The data plane's `HotReloadWatcher` picks up the change
///
/// Returns a `JoinHandle` that can be aborted on shutdown.
pub async fn start(
store: ConfigStore,
ingress_class: String,
namespace: Option<String>,
) -> anyhow::Result<JoinHandle<()>> {
let client = Client::try_default().await.context(
"Failed to create Kubernetes client. Are you running in a cluster or have a kubeconfig?",
)?;
tracing::info!("Kubernetes client initialized");
let store = Arc::new(store);
let client = Arc::new(client);
let reconciler = Arc::new(reconciler::Reconciler::new(store.clone()));
// Callback invoked by every watcher after processing an event.
// This is where cross-referencing happens: routes + certs + endpoints
// are assembled into a complete ProxyConfig.
let on_change: Arc<dyn Fn() + Send + Sync> = {
let r = reconciler.clone();
Arc::new(move || {
r.reconcile();
})
};
let handle = tokio::spawn(async move {
let ingress_handle = ingress_watcher::watch_ingresses(
client.clone(),
store.clone(),
ingress_class,
namespace.clone(),
on_change.clone(),
);
let secret_handle = secret_watcher::watch_secrets(
client.clone(),
store.clone(),
namespace.clone(),
on_change.clone(),
);
let endpoint_handle = endpoint_watcher::watch_endpoints(
client.clone(),
store.clone(),
namespace,
on_change.clone(),
);
tracing::info!("All watchers started");
// If any watcher dies, log the error and attempt restart
tokio::select! {
r = ingress_handle => tracing::error!("Ingress watcher exited: {:?}", r),
r = secret_handle => tracing::error!("Secret watcher exited: {:?}", r),
r = endpoint_handle => tracing::error!("Endpoint watcher exited: {:?}", r),
}
});
Ok(handle)
}

View File

@ -1,241 +0,0 @@
//! Reconcile loop for the GIngress controller.
//!
//! After any watcher detects a change (Ingress, Secret, Endpoints),
//! the reconciler reads all fragments from the ConfigStore, cross-references them,
//! assembles a complete `ProxyConfig`, validates it, and signals a reload.
use gingress_proxy::config::{ConfigStore, Endpoint, ProxyConfig, RouteRule, TlsCert};
use std::collections::{HashMap, HashSet};
use std::sync::Arc;
/// Reconcile the full proxy configuration from current k8s state.
pub struct Reconciler {
store: Arc<ConfigStore>,
}
impl Reconciler {
pub fn new(store: Arc<ConfigStore>) -> Self {
Self { store }
}
/// Trigger a full reconciliation.
///
/// 1. Reads all route fragments (from Ingress watcher)
/// 2. Reads all TLS certs (from Secret watcher)
/// 3. Reads all upstream endpoints (from Endpoint watcher)
/// 4. Cross-references: matches TLS secrets to Ingress hosts,
/// matches upstreams to route backends
/// 5. Validates the configuration
/// 6. Writes the assembled ProxyConfig to the store
/// 7. Signals reload
pub fn reconcile(&self) {
tracing::debug!("Reconciliation started");
// Step 1: Gather all routes from ingress-scoped keys
// Keys look like: "ingress:<ns>/<name>:route:<host>"
let mut routes: HashMap<String, Vec<RouteRule>> = HashMap::new();
for key in self.store.keys_with_prefix("ingress:") {
if !key.contains(":route:") {
continue;
}
// Extract host from "ingress:<ns>/<name>:route:<host>"
if let Some(host) = key.split(":route:").nth(1) {
if let Some(rules) = self.store.get::<Vec<RouteRule>>(&key) {
if !rules.is_empty() {
routes.entry(host.to_string()).or_default().extend(rules);
}
}
}
}
// Step 2: Gather all TLS certs
let mut tls_certs: HashMap<String, TlsCert> = HashMap::new();
for key in self.store.keys_with_prefix("tls:") {
if let Some(cert) = self.store.get::<TlsCert>(&key) {
tls_certs.insert(cert.host.clone(), cert);
}
}
// Step 3: Gather all upstreams keyed by backend ("<ns>/<name>:<port>")
let mut upstreams: HashMap<String, Vec<Endpoint>> = HashMap::new();
for key in self.store.keys_with_prefix("upstream:") {
if let Some(eps) = self.store.get::<Vec<Endpoint>>(&key) {
if !eps.is_empty() {
upstreams.insert(key.clone(), eps);
}
}
}
// Step 4: Gather rate limits, headers, session affinity, and websocket hosts
let rate_limits = self.collect_rate_limits(&routes);
let headers = self.collect_headers();
let session_affinity = self.collect_session_affinity(&routes);
let websocket_hosts = self.collect_websocket_hosts();
let git_upstream = self.collect_git_backend();
// Step 5: Build the complete ProxyConfig
let cfg = ProxyConfig {
routes,
tls: tls_certs,
upstreams,
rate_limits,
headers,
session_affinity,
websocket_hosts,
git_upstream,
};
// Step 6: Validate
let warnings = self.validate_config(&cfg);
for w in &warnings {
tracing::warn!("{}", w);
}
// Step 7: Store the assembled config as the canonical snapshot
self.store.set(
"_assembled",
&serde_json::to_value(&cfg).unwrap_or_default(),
);
self.store.signal_reload();
tracing::info!(
routes = cfg.routes.len(),
tls_hosts = cfg.tls.len(),
upstreams = cfg.upstreams.len(),
warnings = warnings.len(),
"Reconciliation complete"
);
}
/// Cross-reference: for each Ingress TLS entry, find the Secret cert.
///
/// The Ingress TLS section maps: `hosts: [example.com]` → `secretName: my-cert`.
/// The Secret watcher stores the cert at key `tls:<secretName>`.
/// We already map secretName → host in ingress_watcher, so this is a no-op
/// when the ingress_watcher uses correct key mapping.
#[allow(dead_code)]
pub fn cross_reference_tls(&self) -> HashMap<String, TlsCert> {
let mut host_certs: HashMap<String, TlsCert> = HashMap::new();
// TLS secret name → host mapping is stored by the ingress watcher
// at key: "tls-host:<secretName>" → Vec<String> (hosts)
for key in self.store.keys_with_prefix("tls-host:") {
let secret_name = &key["tls-host:".len()..];
let hosts: Vec<String> = self.store.get::<Vec<String>>(&key).unwrap_or_default();
// Look up the actual cert: key "tls:<host>" (stored by secret watcher
// using the certificate's SAN/CN host, but also "tls-secret:<secretName>")
let cert_key = format!("tls-secret:{}", secret_name);
if let Some(cert) = self.store.get::<TlsCert>(&cert_key) {
for host in hosts {
host_certs.insert(host, cert.clone());
}
}
}
host_certs
}
/// Validate the assembled configuration. Returns warnings.
fn validate_config(&self, cfg: &ProxyConfig) -> Vec<String> {
let mut warnings = Vec::new();
// Check: every TLS host has a route
for host in cfg.tls.keys() {
if !cfg.routes.contains_key(host) {
warnings.push(format!(
"TLS configured for host '{}' but no routes exist for this host",
host
));
}
}
// Check: every route backend has upstream endpoints
for (host, rules) in &cfg.routes {
for rule in rules {
let backend_key =
format!("upstream:{}/{}", rule.backend.namespace, rule.backend.name);
if !cfg.upstreams.contains_key(&backend_key) {
warnings.push(format!(
"Host '{}' routes to backend {}/{}:{} but no endpoints found",
host, rule.backend.namespace, rule.backend.name, rule.backend.port
));
}
}
}
// Check: orphaned upstreams (no route references them)
let mut referenced_backends: HashSet<String> = HashSet::new();
for rules in cfg.routes.values() {
for rule in rules {
let bk = format!("upstream:{}/{}", rule.backend.namespace, rule.backend.name);
referenced_backends.insert(bk);
}
}
for upstream_key in cfg.upstreams.keys() {
if !referenced_backends.contains(upstream_key) {
warnings.push(format!(
"Upstream '{}' has no routes referencing it (orphaned)",
upstream_key
));
}
}
warnings
}
/// Collect rate limit policies for all hosts that have routes.
fn collect_rate_limits(
&self,
routes: &HashMap<String, Vec<RouteRule>>,
) -> HashMap<String, gingress_proxy::config::RateLimitPolicy> {
let mut limits = HashMap::new();
for host in routes.keys() {
let key = format!("rate_limit:{}", host);
if let Some(policy) = self.store.get(&key) {
limits.insert(host.clone(), policy);
}
}
limits
}
/// Collect header operations for all hosts.
fn collect_headers(&self) -> HashMap<String, Vec<gingress_proxy::config::HeaderOp>> {
let mut headers = HashMap::new();
for key in self.store.keys_with_prefix("headers:") {
let host = &key["headers:".len()..];
if let Some(ops) = self.store.get(&key) {
headers.insert(host.to_string(), ops);
}
}
headers
}
/// Collect session affinity configs for all hosts that have routes.
fn collect_session_affinity(
&self,
_routes: &HashMap<String, Vec<RouteRule>>,
) -> HashMap<String, gingress_proxy::config::SessionAffinityConfig> {
let mut affinity = HashMap::new();
for key in self.store.keys_with_prefix("session_affinity:") {
let host = &key["session_affinity:".len()..];
if let Some(cfg) = self.store.get(&key) {
affinity.insert(host.to_string(), cfg);
}
}
affinity
}
/// Collect WebSocket-enabled hosts.
fn collect_websocket_hosts(&self) -> Vec<String> {
self.store
.get::<Vec<String>>("websocket:hosts")
.unwrap_or_default()
}
/// Collect git backend configuration from annotation.
fn collect_git_backend(&self) -> Option<gingress_proxy::config::Backend> {
self.store.get("git_backend")
}
}

View File

@ -1,166 +0,0 @@
//! Watches Kubernetes TLS Secrets and loads certificates.
//!
//! Compatible with cert-manager: watches for Secret creation/update events
//! and parses `tls.crt` and `tls.key` into the ConfigStore for TLS termination.
//!
//! Key convention:
//! - `tls-secret:<secretName>` — the raw cert, cross-referenced by reconciler
//! via the `tls-host:<secretName>` mapping written by the ingress watcher.
//! - After reconciliation, the reconciler copies certs to `tls:<host>` for
//! direct SNI lookup by the proxy.
use futures::StreamExt;
use futures::pin_mut;
use gingress_proxy::config::{ConfigStore, TlsCert};
use kube::ResourceExt;
use kube::runtime::watcher::{self, Event};
use std::sync::Arc;
/// Watch Secrets of type `kubernetes.io/tls` and update the ConfigStore.
///
/// After each event, the `on_change` callback is invoked so the reconciler
/// can cross-reference certs with routes.
pub async fn watch_secrets(
client: Arc<kube::Client>,
store: Arc<ConfigStore>,
_namespace: Option<String>,
on_change: Arc<dyn Fn() + Send + Sync>,
) {
let api = kube::Api::<k8s_openapi::api::core::v1::Secret>::all(client.as_ref().clone());
let config = watcher::Config {
field_selector: Some("type=kubernetes.io/tls".to_string()),
..Default::default()
};
let secret_watcher = watcher::watcher(api, config);
pin_mut!(secret_watcher);
while let Some(event) = secret_watcher.next().await {
match event {
Ok(Event::Apply(secret)) => {
process_tls_secret(&secret, &store);
on_change();
tracing::info!(
name = %secret.name_any(),
namespace = %secret.namespace().unwrap_or_default(),
"TLS Secret applied"
);
}
Ok(Event::Init) => {
store.remove_prefix("tls-secret:");
tracing::info!("Secret watcher re-initializing");
}
Ok(Event::InitApply(secret)) => {
process_tls_secret(&secret, &store);
}
Ok(Event::InitDone) => {
store.signal_reload();
on_change();
tracing::info!("Secret watcher init complete");
}
Ok(Event::Delete(secret)) => {
remove_tls_cert(&secret, &store);
on_change();
tracing::info!(
name = %secret.name_any(),
"TLS Secret deleted, cert removed"
);
}
Err(e) => {
tracing::error!("Secret watcher error: {}", e);
}
}
}
}
/// Parse a TLS secret and store the certificate.
fn process_tls_secret(secret: &k8s_openapi::api::core::v1::Secret, store: &ConfigStore) {
let data = match &secret.data {
Some(d) => d,
None => return,
};
let cert_pem = match data
.get("tls.crt")
.and_then(|v| std::str::from_utf8(&v.0).ok())
{
Some(v) => v.to_string(),
None => {
tracing::warn!(name = %secret.name_any(), "TLS Secret missing tls.crt");
return;
}
};
let key_pem = match data
.get("tls.key")
.and_then(|v| std::str::from_utf8(&v.0).ok())
{
Some(v) => v.to_string(),
None => {
tracing::warn!(name = %secret.name_any(), "TLS Secret missing tls.key");
return;
}
};
let secret_name = secret.name_any();
// Extract SANs from the certificate to determine which hosts this cert covers
let hosts = extract_sans_from_pem(&cert_pem).unwrap_or_else(|| vec![secret_name.clone()]);
let tls_cert = TlsCert {
host: hosts.first().cloned().unwrap_or(secret_name.clone()),
cert_pem,
key_pem,
};
// Store under the secret name for cross-referencing
store.set(&format!("tls-secret:{}", secret_name), &tls_cert);
// Also store directly under each SAN host for SNI lookup
for host in &hosts {
store.set(&format!("tls:{}", host), &tls_cert);
}
store.signal_reload();
}
/// Extract Subject Alternative Names from a PEM certificate.
fn extract_sans_from_pem(pem_data: &str) -> Option<Vec<String>> {
use x509_parser::prelude::*;
let mut reader = std::io::BufReader::new(pem_data.as_bytes());
let certs: Vec<_> = rustls_pemfile::certs(&mut reader)
.collect::<Result<Vec<_>, _>>()
.ok()?;
let cert_der = certs.first()?;
let (_, cert) = X509Certificate::from_der(cert_der).ok()?;
let mut hosts: Vec<String> = Vec::new();
if let Ok(Some(san)) = cert.subject_alternative_name() {
for name in &san.value.general_names {
if let GeneralName::DNSName(dns) = name {
hosts.push(dns.to_string());
}
}
}
// Fallback: use CN
if hosts.is_empty() {
if let Some(cn) = cert.subject().iter_common_name().next() {
hosts.push(cn.as_str().unwrap_or_default().to_string());
}
}
if hosts.is_empty() { None } else { Some(hosts) }
}
/// Remove a TLS certificate when the Secret is deleted.
fn remove_tls_cert(secret: &k8s_openapi::api::core::v1::Secret, store: &ConfigStore) {
let secret_name = secret.name_any();
store.remove(&format!("tls-secret:{}", secret_name));
// Also clean up tls-host mapping
store.remove(&format!("tls-host:{}", secret_name));
store.signal_reload();
}

View File

@ -1,174 +0,0 @@
//! GIngress — Kubernetes Ingress Controller
//!
//! Control plane that watches Kubernetes resources (Ingress, Service, Endpoints,
//! Secrets) and updates the shared `ConfigStore` for the data plane.
//!
//! Architecture:
//! - Watches Ingress resources → builds routing rules
//! - Watches TLS Secrets → loads certificates
//! - Watches Endpoints → tracks upstream IPs
//! - Reconciler → diffs changes and pushes to ConfigStore + signals reload
mod controller;
use clap::Parser;
use gingress_proxy::config::ConfigStore;
use gingress_proxy::hot_reload;
use gingress_proxy::observability;
use gingress_proxy::server::{self, GIngressProxy};
#[derive(Parser)]
#[command(name = "gingress")]
struct Args {
/// Ingress class name to watch (default: "gingress")
#[arg(long, default_value = "gingress")]
ingress_class: String,
/// Kubernetes namespace to watch (empty = all namespaces)
#[arg(long)]
namespace: Option<String>,
/// HTTP bind address for the proxy
#[arg(long, default_value = "0.0.0.0:80")]
bind_http: String,
/// HTTPS bind address for the proxy
#[arg(long, default_value = "0.0.0.0:443")]
bind_https: String,
/// Metrics bind address
#[arg(long, default_value = "0.0.0.0:8080")]
metrics_bind: String,
/// Log level
#[arg(long, default_value = "info")]
log_level: String,
/// OTLP endpoint (optional)
#[arg(long)]
otlp_endpoint: Option<String>,
}
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let args = Args::parse();
// Initialize tracing
observability::init_tracing(&args.log_level, args.otlp_endpoint.is_some());
// Initialize OTLP if configured
let _otel_guard = if let Some(ref endpoint) = args.otlp_endpoint {
Some(observability::init_otlp(endpoint, "gingress")?)
} else {
None
};
tracing::info!(
ingress_class = %args.ingress_class,
bind_http = %args.bind_http,
bind_https = %args.bind_https,
"GIngress starting"
);
// Shared config store between control plane and data plane
let config_store = ConfigStore::new();
// Start the control plane: watch k8s resources
let controller_handle = controller::start(
config_store.clone(),
args.ingress_class.clone(),
args.namespace.clone(),
)
.await?;
tracing::info!("Kubernetes controller started");
// Metrics server (for Prometheus scraping)
let metrics_handle = spawn_metrics_server(&args.metrics_bind).await?;
tracing::info!(bind = %args.metrics_bind, "Metrics server started");
// Build the Pingora proxy (data plane)
let proxy = GIngressProxy::new(config_store.clone());
// Spawn hot-reload watcher: applies config changes to the proxy
let reload_handle = hot_reload::spawn_reload_watcher(config_store.clone(), move |store| {
// Read the assembled ProxyConfig that the reconciler wrote at key "_assembled"
match store.get::<serde_json::Value>("_assembled") {
Some(config_json) => {
if let Ok(cfg) =
serde_json::from_value::<gingress_proxy::config::ProxyConfig>(config_json)
{
tracing::info!(
routes = cfg.routes.len(),
tls_hosts = cfg.tls.len(),
upstreams = cfg.upstreams.len(),
"Hot-reload: new proxy configuration applied"
);
// Apply TLS certificates to the proxy
for (_host, cert) in &cfg.tls {
tracing::debug!(
host = %cert.host,
"Hot-reload: TLS cert loaded for host"
);
}
// Apply routes to the proxy
for (host, rules) in &cfg.routes {
tracing::debug!(
host = %host,
num_rules = rules.len(),
"Hot-reload: routes configured"
);
}
} else {
tracing::error!("Hot-reload: failed to deserialize assembled ProxyConfig");
}
}
None => {
tracing::warn!("Hot-reload: no assembled config found (_assembled key missing)");
}
}
});
// Build and run the proxy server (blocking)
let server = server::build_server(proxy, &args.bind_http, &args.bind_https)?;
tracing::info!(
"GIngress proxy starting, listening on {} (HTTP) and {} (HTTPS)",
args.bind_http,
args.bind_https
);
// Run proxy in a tokio blocking task
let proxy_handle = tokio::task::spawn_blocking(move || {
server::run_server(server);
});
// Wait for shutdown signal
tokio::signal::ctrl_c().await?;
tracing::info!("Shutdown signal received, stopping...");
controller_handle.abort();
reload_handle.abort();
metrics_handle.abort();
let _ = tokio::time::timeout(std::time::Duration::from_secs(5), proxy_handle).await;
tracing::info!("GIngress stopped");
Ok(())
}
/// Spawn the metrics server for Prometheus scraping.
async fn spawn_metrics_server(bind: &str) -> anyhow::Result<tokio::task::JoinHandle<()>> {
use std::net::TcpListener;
let bind = bind.to_string();
let listener = TcpListener::bind(&bind)?;
let handle = tokio::spawn(async move {
// Serve metrics via a minimal HTTP handler
// Uses the prometheus_exporter from observability
let _ = listener;
tracing::info!(bind = %bind, "Metrics server stopped");
});
Ok(handle)
}

View File

@ -1,35 +0,0 @@
[package]
name = "git-hook"
version.workspace = true
edition.workspace = true
authors.workspace = true
description.workspace = true
repository.workspace = true
readme.workspace = true
homepage.workspace = true
license.workspace = true
keywords.workspace = true
categories.workspace = true
documentation.workspace = true
[dependencies]
tokio = { workspace = true, features = ["full"] }
git = { workspace = true }
observability = { workspace = true }
db = { workspace = true }
config = { workspace = true }
tracing = { workspace = true }
tracing-subscriber = { workspace = true, features = ["json"] }
anyhow = { workspace = true }
clap = { workspace = true, features = ["derive"] }
tokio-util = { workspace = true }
hyper = { workspace = true }
serde_json = { workspace = true }
sea-orm = { workspace = true }
metrics = "0.22"
metrics-exporter-prometheus = "0.13"
chrono = { workspace = true, features = ["serde"] }
reqwest = { workspace = true }
agent = { workspace = true }
models = { workspace = true }
async-trait = { workspace = true }

View File

@ -1,10 +0,0 @@
use clap::Parser;
#[derive(Parser, Debug)]
#[command(name = "git-hook")]
#[command(version)]
pub struct HookArgs {
/// Worker ID for this instance. Defaults to the HOOK_POOL_WORKER_ID env var or a generated UUID.
#[arg(long)]
pub worker_id: Option<String>,
}

View File

@ -1,256 +0,0 @@
use clap::Parser;
use config::AppConfig;
use db::cache::AppCache;
use db::database::AppDatabase;
use git::hook::HookService;
use git::hook::embed::TagEmbedder;
use metrics::{Unit, describe_counter};
use metrics_exporter_prometheus::PrometheusHandle;
use observability::{HttpMetrics, init_tracing_subscriber, install_recorder, push::MetricsPusher};
use sea_orm::ConnectionTrait;
use std::sync::Arc;
use tokio::signal;
mod args;
use args::HookArgs;
/// Initialize EmbedService from config (graceful degradation).
async fn init_embed_service(
cfg: &AppConfig,
db: &AppDatabase,
) -> Result<agent::embed::EmbedService, Box<dyn std::error::Error + Send + Sync>> {
let client = agent::new_embed_client(cfg).await?;
let model_name = cfg
.get_embed_model_name()
.unwrap_or_else(|_| "text-embedding-3-small".into());
let dimensions = cfg.get_embed_model_dimensions().unwrap_or(1536);
let svc = agent::embed::EmbedService::new(client, db.writer().clone(), model_name, dimensions);
let _ = svc.ensure_collections().await;
tracing::info!("hook worker: EmbedService initialized for tag embedding");
Ok(svc)
}
/// Adapter that wraps agent's EmbedService to implement git's TagEmbedder trait.
struct EmbedServiceAdapter(agent::embed::EmbedService);
#[async_trait::async_trait]
impl TagEmbedder for EmbedServiceAdapter {
async fn embed_tags_batch(
&self,
tags: Vec<models::TagEmbedInput>,
) -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
// Convert from models::TagEmbedInput to agent's TagEmbedInput (same struct, different path)
let agent_tags: Vec<agent::embed::TagEmbedInput> = tags
.into_iter()
.map(|t| agent::embed::TagEmbedInput {
repo_id: t.repo_id,
repo_name: t.repo_name,
project_id: t.project_id,
name: t.name,
description: t.description,
})
.collect();
self.0
.embed_tags_batch(agent_tags)
.await
.map_err(|e| Box::new(e) as Box<dyn std::error::Error + Send + Sync>)
}
}
async fn http_handler(
db: Arc<AppDatabase>,
cache: Arc<AppCache>,
metrics: Arc<PrometheusHandle>,
req: hyper::Request<hyper::Body>,
) -> Result<hyper::Response<hyper::Body>, std::convert::Infallible> {
match req.uri().path() {
"/health" => {
let writer_ok = db.writer().execute_unprepared("SELECT 1").await.is_ok();
let reader_ok = db.reader().execute_unprepared("SELECT 1").await.is_ok();
let db_ok = writer_ok && reader_ok;
let cache_ok = cache.conn().await.is_ok();
let body = serde_json::json!({
"status": if db_ok && cache_ok { "ok" } else { "unhealthy" },
"db": if db_ok { "ok" } else { "error" },
"cache": if cache_ok { "ok" } else { "error" },
});
let status = if db_ok && cache_ok { 200 } else { 503 };
let body_bytes = match serde_json::to_string(&body) {
Ok(s) => hyper::Body::from(s),
Err(e) => {
return Ok(hyper::Response::builder()
.status(500)
.body(hyper::Body::from(format!("serialize error: {}", e)))
.expect("static response"));
}
};
Ok(hyper::Response::builder()
.status(status)
.header("content-type", "application/json")
.body(body_bytes)
.expect("static response"))
}
"/metrics" => {
let body = metrics.render();
Ok(hyper::Response::builder()
.status(200)
.header("content-type", "text/plain; version=0.0.4; charset=utf-8")
.body(hyper::Body::from(body))
.expect("static response"))
}
_ => Ok(hyper::Response::builder()
.status(404)
.body(hyper::Body::from("not found"))
.expect("static response")),
}
}
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let cfg = AppConfig::load();
let log_level = cfg.log_level().unwrap_or_else(|_| "info".to_string());
init_tracing_subscriber(&log_level, false);
// Pre-register all hook metrics so they appear in /metrics even before first increment.
describe_counter!("hook_tasks_total", Unit::Count, "Total hook tasks dequeued");
describe_counter!(
"hook_tasks_success_total",
Unit::Count,
"Hook tasks completed successfully"
);
describe_counter!(
"hook_tasks_failed_total",
Unit::Count,
"Hook tasks that failed"
);
describe_counter!(
"hook_tasks_locked_total",
Unit::Count,
"Hook tasks re-queued due to repo lock"
);
describe_counter!(
"hook_tasks_retried_total",
Unit::Count,
"Hook tasks that entered retry"
);
describe_counter!(
"hook_tasks_exhausted_total",
Unit::Count,
"Hook tasks that exhausted retries"
);
describe_counter!(
"hook_sync_branches_changed_total",
Unit::Count,
"Branches changed during sync"
);
describe_counter!(
"hook_sync_tags_changed_total",
Unit::Count,
"Tags changed during sync"
);
let metrics_handle = Arc::new(install_recorder());
let http_metrics = Arc::new(HttpMetrics::new()); // Worker app — HTTP section will be empty
// Metrics pusher: periodically push all metrics to apps/metrics aggregator
if let Some(push_url) = std::env::var("METRICS_PUSH_URL").ok() {
let pusher = MetricsPusher::new(&push_url, "git-hook");
pusher.spawn(
http_metrics.clone(),
metrics_handle.clone(),
std::time::Duration::from_secs(15),
);
tracing::info!(push_url = %push_url, "Metrics pusher started (interval 15s)");
}
let db = Arc::new(AppDatabase::init(&cfg).await?);
tracing::info!("database connected");
// 4. Connect to Redis cache (also provides the cluster pool for hook queue)
let cache = Arc::new(AppCache::init(&cfg).await?);
tracing::info!("cache connected");
// 5. Parse CLI args
let _args = HookArgs::parse();
tracing::info!("git-hook worker starting");
// 6. Build and start git hook service
let mut hooks = HookService::new(
(*db).clone(),
(*cache).clone(),
cache.redis_pool().clone(),
cfg.clone(),
);
// Optionally initialize tag embedding
if let Ok(embed_svc) = init_embed_service(&cfg, &db).await {
let adapter = EmbedServiceAdapter(embed_svc);
hooks = hooks.with_tag_embedder(Arc::new(adapter));
}
let cancel = hooks.start_worker().await;
let cancel_signal = cancel.clone();
// 7. Start health/metrics server on a dedicated port
let health_db = db.clone();
let health_cache = cache.clone();
let health_metrics = metrics_handle.clone();
let health_addr: std::net::SocketAddr = ([0, 0, 0, 0], 8083).into();
let health_service = hyper::service::make_service_fn(move |_| {
let db = health_db.clone();
let cache = health_cache.clone();
let metrics = health_metrics.clone();
let service = hyper::service::service_fn(move |req| {
http_handler(db.clone(), cache.clone(), metrics.clone(), req)
});
async move { Ok::<_, std::convert::Infallible>(service) }
});
let health_server = hyper::Server::bind(&health_addr).serve(health_service);
tracing::info!(port = 8083, "health/metrics server started");
tokio::spawn(async move {
if let Err(e) = health_server.await {
tracing::error!("health check server error: {}", e);
}
});
// Spawn signal handler that cancels on SIGINT/SIGTERM
tokio::spawn(async move {
let ctrl_c = async {
signal::ctrl_c()
.await
.expect("failed to install CTRL+C handler");
};
#[cfg(unix)]
let term = async {
use tokio::signal::unix::{SignalKind, signal};
let mut sig =
signal(SignalKind::terminate()).expect("failed to install SIGTERM handler");
sig.recv().await;
};
#[cfg(not(unix))]
let term = std::future::pending::<()>();
tokio::select! {
_ = ctrl_c => {
tracing::info!("received SIGINT, initiating shutdown");
}
_ = term => {
tracing::info!("received SIGTERM, initiating shutdown");
}
}
cancel_signal.cancel();
});
// Wait until the worker is cancelled (by signal handler or otherwise)
cancel.cancelled().await;
tracing::info!("git-hook worker stopped");
Ok(())
}

View File

@ -1,31 +0,0 @@
[package]
name = "gitserver"
version.workspace = true
edition.workspace = true
authors.workspace = true
description.workspace = true
repository.workspace = true
readme.workspace = true
homepage.workspace = true
license.workspace = true
keywords.workspace = true
categories.workspace = true
documentation.workspace = true
[[bin]]
name = "gitserver"
path = "src/main.rs"
[dependencies]
tokio = { workspace = true, features = ["full"] }
git = { workspace = true }
observability = { workspace = true }
tracing = { workspace = true }
db = { workspace = true }
config = { workspace = true }
anyhow = { workspace = true }
clap = { workspace = true, features = ["derive"] }
chrono = { workspace = true, features = ["serde"] }
[lints]
workspace = true

View File

@ -1,59 +0,0 @@
use clap::Parser;
use config::AppConfig;
use observability::{HttpMetrics, init_tracing_subscriber, install_recorder, push::MetricsPusher};
use std::sync::Arc;
#[derive(Parser, Debug)]
#[command(name = "gitserver")]
#[command(version)]
struct Args {
#[arg(long, default_value = "info")]
log_level: String,
}
#[tokio::main]
async fn main() -> anyhow::Result<()> {
let args = Args::parse();
let cfg = AppConfig::load();
init_tracing_subscriber(&args.log_level, false);
let prometheus_handle = Arc::new(install_recorder());
let http_metrics = Arc::new(HttpMetrics::new());
// Metrics pusher: periodically push all metrics to apps/metrics aggregator
if let Some(push_url) = std::env::var("METRICS_PUSH_URL").ok() {
let pusher = MetricsPusher::new(&push_url, "gitserver");
pusher.spawn(
http_metrics.clone(),
prometheus_handle.clone(),
std::time::Duration::from_secs(15),
);
tracing::info!(push_url = %push_url, "Metrics pusher started (interval 15s)");
}
let http_handle = tokio::spawn(git::http::run_http(cfg.clone()));
let ssh_handle = tokio::spawn(git::ssh::run_ssh(cfg));
tokio::select! {
result = http_handle => {
match result {
Ok(Ok(())) => tracing::info!("HTTP server stopped"),
Ok(Err(e)) => tracing::error!("HTTP server error: {}", e),
Err(e) => tracing::error!("HTTP server task panicked: {}", e),
}
}
result = ssh_handle => {
match result {
Ok(Ok(())) => tracing::info!("SSH server stopped"),
Ok(Err(e)) => tracing::error!("SSH server error: {}", e),
Err(e) => tracing::error!("SSH server task panicked: {}", e),
}
}
_ = tokio::signal::ctrl_c() => {
tracing::info!("received shutdown signal");
}
}
tracing::info!("shutting down");
Ok(())
}

View File

@ -1,58 +0,0 @@
[package]
name = "metrics-aggregator"
version.workspace = true
edition.workspace = true
authors.workspace = true
description = "Unified observability aggregator: scrapes metrics, forwards traces, collects logs"
repository.workspace = true
readme.workspace = true
homepage.workspace = true
license.workspace = true
keywords.workspace = true
categories.workspace = true
documentation.workspace = true
[[bin]]
name = "metrics-aggregator"
path = "src/main.rs"
[dependencies]
tokio = { workspace = true, features = ["full"] }
config = { workspace = true }
tracing = { workspace = true }
tracing-subscriber = { workspace = true, features = ["env-filter", "json"] }
observability = { workspace = true }
anyhow = { workspace = true }
clap = { workspace = true, features = ["derive", "env"] }
serde_json = { workspace = true }
chrono = { workspace = true, features = ["serde"] }
serde = { workspace = true, features = ["derive"] }
# HTTP server
actix-web = "4.13.0"
actix-rt = "2.11.0"
# HTTP client for scraping (uses awc = actix-web client, no extra TLS deps)
awc = { workspace = true }
# HTTP client for Loki (reqwest is Send+Sync, unlike awc::Client)
reqwest = { workspace = true, features = ["json"] }
# Metrics
metrics = { workspace = true }
metrics-exporter-prometheus = { version = "0.18", default-features = false, features = ["http-listener", "tokio"] }
# Observability
opentelemetry = { workspace = true }
opentelemetry_sdk = { workspace = true }
opentelemetry-otlp = { version = "0.31.0", default-features = false, features = ["http-proto", "tokio", "trace", "tonic"] }
tracing-opentelemetry = "0.32.1"
tokio-util = { workspace = true }
tokio-stream = { workspace = true }
futures = { workspace = true }
url = { workspace = true }
tower = { workspace = true }
[lints]
workspace = true

View File

@ -1,35 +0,0 @@
use clap::Parser;
#[derive(Parser, Debug)]
#[command(name = "metrics-aggregator")]
#[command(version)]
pub struct Args {
#[arg(long, default_value = "9090", env = "METRICS_AGGREGATOR_PORT")]
pub port: u16,
#[arg(long, env = "OTEL_EXPORTER_OTLP_ENDPOINT")]
pub otel_endpoint: Option<String>,
#[arg(long, env = "LOKI_URL")]
pub loki_url: Option<String>,
#[arg(long, default_value = "15", env = "SCRAPE_INTERVAL_SECS")]
pub scrape_interval_secs: u64,
/// JSON file with scrape targets.
#[arg(long, env = "SCRAPE_TARGETS_FILE")]
pub targets_file: Option<String>,
#[arg(long, default_value = "info", env = "LOG_LEVEL")]
pub log_level: String,
/// Comma-separated list of app names to scrape.
#[arg(long, env = "SCRAPE_APPS")]
pub scrape_apps: Option<String>,
#[arg(long)]
pub no_otel: bool,
#[arg(long)]
pub no_loki: bool,
}

View File

@ -1,40 +0,0 @@
use std::sync::Arc;
use tokio::sync::RwLock;
use crate::target::{ScrapeTarget, load_targets_from_file};
pub async fn watch_targets_file(
path: String,
targets: Arc<RwLock<Vec<ScrapeTarget>>>,
mut shutdown: tokio::sync::broadcast::Receiver<()>,
) {
let mtime_path = path;
let mut last_mtime: Option<std::time::SystemTime> = None;
loop {
tokio::select! {
_ = shutdown.recv() => break,
_ = tokio::time::sleep(std::time::Duration::from_secs(10)) => {
let metadata = match tokio::fs::metadata(&mtime_path).await {
Ok(m) => m,
Err(_) => continue,
};
let current_mtime = metadata.modified().ok();
if current_mtime != last_mtime {
last_mtime = current_mtime;
match load_targets_from_file(&mtime_path).await {
Ok(new_targets) => {
let mut guard = targets.write().await;
*guard = new_targets;
tracing::info!(path = %mtime_path, "targets file reloaded");
}
Err(e) => {
tracing::warn!(error = %e, "failed to reload targets file");
}
}
}
}
}
}
}

View File

@ -1,70 +0,0 @@
use std::time::Duration;
use awc::Client;
use crate::target::ScrapeTarget;
pub async fn k8s_pod_discovery() -> Option<Vec<ScrapeTarget>> {
let pod_namespace = std::env::var("POD_NAMESPACE").ok()?;
let token_path = "/var/run/secrets/kubernetes.io/serviceaccount/token";
let token = tokio::fs::read_to_string(token_path).await.ok()?;
let client = Client::builder()
.timeout(Duration::from_secs(5))
.add_default_header((
awc::http::header::AUTHORIZATION.as_str(),
format!("Bearer {}", token),
))
.finish();
let api_url = format!(
"https://kubernetes.default.svc/api/v1/namespaces/{}/pods",
pod_namespace
);
let mut response = client.get(api_url).send().await.ok()?;
let body_bytes = response.body().await.ok()?;
let pod_list: serde_json::Value = serde_json::from_slice(&body_bytes).ok()?;
let targets: Vec<ScrapeTarget> = pod_list["items"]
.as_array()?
.iter()
.filter_map(|pod| {
let name = pod["metadata"]["name"].as_str()?.to_string();
let phase = pod["status"]["phase"].as_str()?;
if phase != "Running" {
return None;
}
let pod_ip = pod["status"]["podIP"].as_str()?;
let annotations = pod["metadata"]["annotations"].as_object()?;
let port: u16 = annotations
.get("metrics.port")
.and_then(|v| v.as_str())
.and_then(|s| s.parse().ok())
.unwrap_or(8080);
let path = annotations
.get("metrics.path")
.and_then(|v| v.as_str())
.unwrap_or("/metrics");
let labels = pod["metadata"]["labels"]
.as_object()
.map(|m| {
m.iter()
.filter_map(|(k, v)| v.as_str().map(|s| (k.clone(), s.to_string())))
.collect()
})
.unwrap_or_default();
Some(ScrapeTarget {
name,
addr: format!("{}:{}", pod_ip, port),
metrics_path: path.to_string(),
labels,
})
})
.collect();
Some(targets)
}

View File

@ -1,70 +0,0 @@
use chrono::{DateTime, Utc};
use reqwest::Client;
use serde::Serialize;
use std::collections::HashMap;
#[derive(Clone)]
pub struct LokiForwarder {
url: String,
client: Client,
labels: HashMap<String, String>,
}
impl LokiForwarder {
pub fn new(url: String) -> Self {
Self {
url,
client: Client::builder()
.timeout(std::time::Duration::from_secs(5))
.build()
.expect("valid reqwest client"),
labels: HashMap::new(),
}
}
pub async fn push(&self, log_entries: Vec<LokiEntry>) -> anyhow::Result<()> {
if log_entries.is_empty() {
return Ok(());
}
let streams: Vec<LokiStream> = vec![LokiStream {
stream: self.labels.clone(),
values: log_entries
.into_iter()
.map(|e| (format!("{}", e.timestamp), e.line))
.collect(),
}];
let payload = LokiPayload { streams };
let resp = self
.client
.post(&self.url)
.header("Content-Type", "application/json")
.json(&payload)
.send()
.await;
match resp {
Ok(r) if r.status().is_success() => Ok(()),
Ok(r) => anyhow::bail!("Loki push failed: {}", r.status()),
Err(e) => anyhow::bail!("Loki push error: {}", e),
}
}
}
#[derive(Serialize)]
struct LokiPayload {
streams: Vec<LokiStream>,
}
#[derive(Serialize)]
struct LokiStream {
stream: HashMap<String, String>,
values: Vec<(String, String)>,
}
pub struct LokiEntry {
pub timestamp: DateTime<Utc>,
pub line: String,
}

View File

@ -1,633 +0,0 @@
//! Unified observability aggregator for in-cluster deployment.
//!
//! Collects metrics from all app pods via Prometheus scrape, forwards traces
//! to OTLP endpoint, and streams logs from all pods to Loki-compatible backend.
//!
//! Usage:
//! METRICS_AGGREGATOR_PORT=9090 \
//! OTEL_EXPORTER_OTLP_ENDPOINT=http://otel-collector:4317 \
//! LOKI_URL=http://loki:3100/loki/api/v1/push \
//! SCRAPE_INTERVAL_SECS=15 \
//! SCRAPE_TARGETS_FILE=/etc/metrics/targets.json \
//! metrics-aggregator
mod args;
mod hotreload;
mod k8s_discovery;
mod loki;
mod metrics;
mod otel;
mod scrape;
mod stats_store;
mod target;
use serde::Deserialize;
use std::collections::HashMap;
use std::fmt::Write as _;
use std::net::SocketAddr;
use std::sync::Arc;
use std::time::Duration;
use actix_web::{HttpResponse, HttpServer, web};
use clap::Parser;
use loki::{LokiEntry, LokiForwarder};
use metrics::AggMetrics;
use observability::{init_tracing_subscriber, install_recorder, instance_id};
use otel::OtelGuard;
use scrape::{HttpClient, ScrapeResult};
use stats_store::StatsStore;
use target::ScrapeTarget;
use tokio::io::AsyncBufReadExt;
use tokio::sync::{RwLock, broadcast};
use tokio::time::interval;
type MetricsStore = Arc<RwLock<HashMap<String, Vec<scrape::PromMetric>>>>;
// StatsStore is defined in stats_store.rs — per-app aggregated data.
#[actix_web::main]
async fn main() -> std::io::Result<()> {
let args = args::Args::parse();
init_tracing_subscriber(&args.log_level, false);
let instance = instance_id();
tracing::info!(
instance = %instance,
port = args.port,
scrape_interval = args.scrape_interval_secs,
"metrics-aggregator starting"
);
let prometheus_handle = install_recorder();
metrics::init();
let metrics = AggMetrics::new();
let store: MetricsStore = Arc::new(RwLock::new(HashMap::new()));
let stats_store: StatsStore = Arc::new(RwLock::new(HashMap::new()));
let targets: Arc<RwLock<Vec<ScrapeTarget>>> = Arc::new(RwLock::new(Vec::new()));
let http = HttpClient::new(10);
let otel_guard = init_otel_from_args(&args);
let loki = init_loki_from_args(&args);
let (shutdown_tx, _) = broadcast::channel::<()>(4);
// Background task: evict push entries older than 5 minutes.
let stats_store_for_evict = stats_store.clone();
let mut evict_shutdown = shutdown_tx.subscribe();
tokio::spawn(async move {
let mut ticker = interval(Duration::from_secs(30));
loop {
tokio::select! {
_ = evict_shutdown.recv() => break,
_ = ticker.tick() => {
let cutoff = chrono::Utc::now().timestamp() - 300;
let mut guard = stats_store_for_evict.write().await;
guard.retain(|_, entry| entry.last_seen >= cutoff);
}
}
}
});
if let Some(path) = &args.targets_file {
match target::load_targets_from_file(path).await {
Ok(initial_targets) => {
let mut guard = targets.write().await;
*guard = initial_targets;
tracing::info!(count = guard.len(), "loaded initial targets from file");
}
Err(e) => {
tracing::warn!(error = %e, "failed to load targets file");
}
}
let tw =
hotreload::watch_targets_file(path.clone(), targets.clone(), shutdown_tx.subscribe());
tokio::spawn(tw);
} else if std::env::var("KUBERNETES_SERVICE_HOST").is_ok() {
if let Some(k8s_targets) = k8s_discovery::k8s_pod_discovery().await {
let mut guard = targets.write().await;
*guard = k8s_targets.clone();
tracing::info!(count = guard.len(), "discovered K8s pods as targets");
}
}
let scrape_filter = args
.scrape_apps
.as_ref()
.map(|s| s.split(',').map(|p| p.trim().to_string()).collect());
let scrape_targets = targets.clone();
let scrape_store = store.clone();
let scrape_metrics = metrics.clone();
let scrape_http = http.clone();
let loki_clone = loki.clone();
let shutdown_tx_clone = shutdown_tx.clone();
let scrape_interval = args.scrape_interval_secs;
let scrape_filter_clone = scrape_filter.clone();
tokio::task::spawn_local(async move {
scrape_loop(
scrape_targets,
scrape_store,
scrape_metrics,
scrape_http,
scrape_interval,
scrape_filter_clone,
loki_clone,
shutdown_tx_clone.subscribe(),
)
.await;
});
let log_shutdown = shutdown_tx.subscribe();
let log_loki = loki.clone();
tokio::task::spawn_local(async move {
log_collector(log_loki, log_shutdown).await;
});
let bind_addr: SocketAddr = ([0, 0, 0, 0], args.port).into();
tracing::info!(addr = %bind_addr, "HTTP server starting");
let app_targets = targets.clone();
let app_store = store.clone();
let app_handle = prometheus_handle.clone();
let loki_for_push: Option<Arc<LokiForwarder>> = loki.map(Arc::new);
let app_stats = stats_store.clone();
let server = HttpServer::new(move || {
let targets = app_targets.clone();
let store = app_store.clone();
let handle = app_handle.clone();
let stats_store = app_stats.clone();
let loki_for_push: Option<Arc<LokiForwarder>> = loki_for_push.clone();
actix_web::App::new()
.app_data(web::Data::new(targets))
.app_data(web::Data::new(store))
.app_data(web::Data::new(handle))
.app_data(web::Data::new(stats_store))
.app_data(web::Data::new(loki_for_push))
.route("/metrics", web::get().to(handle_metrics))
.route("/api/v1/metrics", web::get().to(handle_metrics))
.route("/api/v1/push", web::post().to(handle_push))
.route("/api/v1/dashboard", web::get().to(handle_dashboard))
.route("/api/v1/stats", web::get().to(handle_stats))
.route("/health", web::get().to(handle_health))
.route("/api/v1/health", web::get().to(handle_health))
.route("/api/v1/targets", web::get().to(handle_targets))
})
.bind(&bind_addr)?
.run();
let server_handle = server.handle();
tokio::spawn(server);
tokio::signal::ctrl_c().await.ok();
tracing::info!("received Ctrl+C, shutting down");
let _ = shutdown_tx.send(());
server_handle.stop(true).await;
if let Some(guard) = otel_guard {
guard.shutdown().await;
}
tracing::info!("metrics-aggregator stopped");
Ok(())
}
fn init_otel_from_args(args: &args::Args) -> Option<OtelGuard> {
if args.no_otel {
return None;
}
let endpoint = args
.otel_endpoint
.clone()
.or_else(|| std::env::var("OTEL_EXPORTER_OTLP_ENDPOINT").ok())?;
match otel::init_otel(&endpoint, "metrics-aggregator") {
Ok(guard) => {
tracing::info!(endpoint = %endpoint, "OTLP tracing enabled");
Some(guard)
}
Err(e) => {
tracing::warn!(error = %e, "OTLP init failed, continuing without traces");
None
}
}
}
fn init_loki_from_args(args: &args::Args) -> Option<LokiForwarder> {
if args.no_loki {
return None;
}
let url = args
.loki_url
.clone()
.or_else(|| std::env::var("LOKI_URL").ok())?;
tracing::info!("Loki log forwarding enabled");
Some(LokiForwarder::new(url))
}
async fn handle_metrics(
store: web::Data<MetricsStore>,
stats_store: web::Data<StatsStore>,
handle: web::Data<observability::PrometheusHandle>,
) -> HttpResponse {
let extra = vec![("aggregator_instance".to_string(), "default".to_string())];
let scraped = render_aggregated_metrics(store, extra.clone()).await;
let pushed = render_pushed_metrics(stats_store).await;
let combined = format!("{}{}{}", handle.render(), scraped, pushed);
HttpResponse::Ok()
.content_type("text/plain; version=0.0.4; charset=utf-8")
.body(combined)
}
async fn handle_health() -> HttpResponse {
HttpResponse::Ok()
.content_type("application/json")
.body(r#"{"status":"ok"}"#)
}
async fn handle_targets(targets: web::Data<Arc<RwLock<Vec<ScrapeTarget>>>>) -> HttpResponse {
let guard = targets.read().await;
let json = serde_json::to_string(&*guard).unwrap_or_default();
HttpResponse::Ok()
.content_type("application/json")
.body(json)
}
// ── Push endpoint payload ────────────────────────────────────────────────────
#[derive(Debug, Deserialize)]
#[serde(rename_all = "camelCase")]
struct PushPayload {
app: String,
#[serde(default)]
instance: String,
timestamp: i64,
#[serde(default)]
http: Option<observability::push::HttpPayload>,
#[serde(default)]
system: Option<observability::push::SystemPayload>,
#[serde(default)]
business: HashMap<String, f64>,
#[serde(default)]
token_usage: Option<observability::push::TokenUsagePayload>,
#[serde(default)]
tasks: Option<observability::push::TaskStatsPayload>,
#[serde(default)]
latency: HashMap<String, observability::push::LatencySnapshot>,
#[serde(default)]
logs: Vec<observability::push::LogEntry>,
}
async fn handle_push(
stats_store: web::Data<StatsStore>,
loki: web::Data<Option<Arc<LokiForwarder>>>,
payload: web::Json<PushPayload>,
) -> HttpResponse {
let app = payload.app.clone();
stats_store::merge_push_payload(
&stats_store,
&app,
&payload.instance,
payload.timestamp,
payload.http.as_ref(),
payload.system.as_ref(),
&payload.business,
payload.token_usage.as_ref(),
payload.tasks.as_ref(),
&payload.latency,
&payload.logs,
)
.await;
// Forward logs to Loki if configured
if !payload.logs.is_empty() {
if let Some(loki_fwd) = loki.as_ref() {
let entries: Vec<LokiEntry> = payload
.logs
.iter()
.map(|l| LokiEntry {
timestamp: chrono::DateTime::from_timestamp(l.timestamp, 0)
.unwrap_or_else(chrono::Utc::now),
line: format!("[{}] {}", l.level.to_lowercase(), l.message),
})
.collect();
if let Err(e) = loki_fwd.push(entries).await {
tracing::warn!(error = %e, "loki push on /push failed");
}
}
}
HttpResponse::Ok().body("ok")
}
async fn scrape_loop(
targets: Arc<RwLock<Vec<ScrapeTarget>>>,
store: MetricsStore,
metrics: AggMetrics,
http: HttpClient,
interval_secs: u64,
scrape_apps_filter: Option<Vec<String>>,
_loki: Option<LokiForwarder>,
mut shutdown: broadcast::Receiver<()>,
) {
let mut ticker = interval(Duration::from_secs(interval_secs));
loop {
tokio::select! {
_ = shutdown.recv() => break,
_ = ticker.tick() => {
let targets_snapshot = targets.read().await.clone();
let count = targets_snapshot.len() as u64;
metrics.targets_total.set(count as f64);
let mut healthy_count = 0u64;
for target in &targets_snapshot {
if let Some(ref filter) = scrape_apps_filter {
if !filter.contains(&target.name) {
continue;
}
}
metrics.scrape_total.increment(1);
match http.scrape(target).await {
ScrapeResult::Success(body, duration_ms) => {
metrics.scrape_success.increment(1);
metrics.scrape_duration.record(duration_ms);
let parsed = scrape::parse_prometheus(&body);
update_store(store.clone(), &target.name, parsed).await;
healthy_count += 1;
}
ScrapeResult::Timeout => {
metrics.scrape_failures.increment(1);
metrics.scrape_errors_timeout.increment(1);
tracing::warn!(target = %target.name, "scrape timeout");
}
ScrapeResult::ConnectionError(e) => {
metrics.scrape_failures.increment(1);
metrics.scrape_errors_connection.increment(1);
tracing::warn!(target = %target.name, error = %e, "scrape connection error");
}
ScrapeResult::HttpError(status) => {
metrics.scrape_failures.increment(1);
tracing::warn!(target = %target.name, status = status, "scrape HTTP error");
}
}
}
metrics.targets_healthy.set(healthy_count as f64);
}
}
}
}
async fn update_store(store: MetricsStore, target_name: &str, metrics: Vec<scrape::PromMetric>) {
let mut guard = store.write().await;
guard.insert(target_name.to_string(), metrics);
}
async fn render_aggregated_metrics(
store: web::Data<MetricsStore>,
extra_group_labels: Vec<(String, String)>,
) -> String {
let guard = store.read().await;
let mut output = String::new();
for (target_name, metrics) in guard.iter() {
for metric in metrics {
let mut labels = metric.labels.clone();
labels.insert(
"aggregated_by".to_string(),
"metrics-aggregator".to_string(),
);
labels.insert("source_target".to_string(), target_name.clone());
for (k, v) in &extra_group_labels {
labels.insert(k.clone(), v.clone());
}
let label_str = if labels.is_empty() {
String::new()
} else {
let pairs: Vec<String> = labels
.iter()
.map(|(k, v)| {
format!(
r#"{}="{}""#,
k,
v.replace('\\', "\\\\").replace('"', "\\\"")
)
})
.collect();
format!("{{{}}}", pairs.join(","))
};
let _ = writeln!(&mut output, "{}{} {}", metric.name, label_str, metric.value);
}
}
output
}
async fn render_pushed_metrics(stats_store: web::Data<StatsStore>) -> String {
let guard = stats_store.read().await;
let mut output = String::new();
for (app_name, entry) in guard.iter() {
let labels = [
format!(r#"app="{}""#, app_name),
"aggregated_by".to_string(),
"metrics-aggregator".to_string(),
"push_source=true".to_string(),
];
let label_str = format!("{{{}}}", labels.join(","));
let h = &entry;
let _ = writeln!(
&mut output,
"push_http_requests_total{} {}",
label_str, h.requests_total
);
let _ = writeln!(
&mut output,
"push_http_request_duration_ms_total{} {}",
label_str, h.request_duration_ms_total
);
let _ = writeln!(
&mut output,
"push_http_requests_2xx{} {}",
label_str, h.requests_2xx
);
let _ = writeln!(
&mut output,
"push_http_requests_4xx{} {}",
label_str, h.requests_4xx
);
let _ = writeln!(
&mut output,
"push_http_requests_5xx{} {}",
label_str, h.requests_5xx
);
for (endpoint, &count) in &h.endpoints {
let sanitized = endpoint.replace([' ', '/'], "_").to_lowercase();
let ep_labels = format!(
r#"app="{}",endpoint="{}",aggregated_by="metrics-aggregator",push_source="true""#,
app_name, sanitized
);
let _ = writeln!(
&mut output,
"push_http_endpoint_requests_total{{{}}} {}",
ep_labels, count
);
}
// System metrics in Prometheus format
let sys_labels = format!(r#"app="{}",aggregated_by="metrics-aggregator""#, app_name);
let _ = writeln!(
&mut output,
"system_cpu_usage_percent{{{}}} {}",
sys_labels, h.cpu_usage_percent
);
let _ = writeln!(
&mut output,
"system_memory_used_mb{{{}}} {}",
sys_labels, h.memory_used_mb
);
let _ = writeln!(
&mut output,
"system_memory_total_mb{{{}}} {}",
sys_labels, h.memory_total_mb
);
let _ = writeln!(
&mut output,
"system_uptime_secs{{{}}} {}",
sys_labels, h.uptime_secs
);
// Business counters
for (counter_name, value) in &h.business {
let biz_labels = format!(r#"app="{}",aggregated_by="metrics-aggregator""#, app_name);
let _ = writeln!(&mut output, "{}{{{}}} {}", counter_name, biz_labels, value);
}
// Token usage
let ai_labels = format!(r#"app="{}",aggregated_by="metrics-aggregator""#, app_name);
let _ = writeln!(
&mut output,
"ai_input_tokens_total{{{}}} {}",
ai_labels, h.ai_input_tokens_total
);
let _ = writeln!(
&mut output,
"ai_output_tokens_total{{{}}} {}",
ai_labels, h.ai_output_tokens_total
);
let _ = writeln!(
&mut output,
"ai_calls_total{{{}}} {}",
ai_labels, h.ai_calls_total
);
// Latency per endpoint
for (endpoint, lat) in &h.latency {
let lat_labels = format!(
r#"app="{}",endpoint="{}",aggregated_by="metrics-aggregator""#,
app_name, endpoint
);
let _ = writeln!(
&mut output,
"latency_p99_ms{{{}}} {}",
lat_labels, lat.p99_ms
);
let _ = writeln!(
&mut output,
"latency_p90_ms{{{}}} {}",
lat_labels, lat.p90_ms
);
let _ = writeln!(
&mut output,
"latency_p50_ms{{{}}} {}",
lat_labels, lat.p50_ms
);
let _ = writeln!(
&mut output,
"latency_max_ms{{{}}} {}",
lat_labels, lat.max_ms
);
}
}
output
}
// ── JSON API handlers ────────────────────────────────────────────────────────
async fn handle_dashboard(stats_store: web::Data<StatsStore>) -> HttpResponse {
let dashboard = stats_store::build_dashboard(&stats_store).await;
let json = serde_json::to_string(&dashboard).unwrap_or_default();
HttpResponse::Ok()
.content_type("application/json")
.body(json)
}
async fn handle_stats(stats_store: web::Data<StatsStore>) -> HttpResponse {
// Returns per-app stats as JSON
let guard = stats_store.read().await;
let json = serde_json::to_string(&*guard).unwrap_or_default();
HttpResponse::Ok()
.content_type("application/json")
.body(json)
}
async fn log_collector(loki: Option<LokiForwarder>, mut shutdown: broadcast::Receiver<()>) {
let stdin = tokio::io::stdin();
let mut reader = tokio::io::BufReader::new(stdin);
let mut interval_tick = interval(Duration::from_secs(1));
let mut batch: Vec<LokiEntry> = Vec::with_capacity(100);
let mut line_buf = String::new();
loop {
tokio::select! {
_ = shutdown.recv() => break,
_ = interval_tick.tick() => {
if !batch.is_empty() {
if let Some(ref loki) = loki {
if let Err(e) = loki.push(std::mem::take(&mut batch)).await {
tracing::warn!(error = %e, "Loki push failed");
}
}
}
}
_ = async { line_buf.clear(); reader.read_line(&mut line_buf).await.ok() } => {
if !line_buf.is_empty() {
let line = line_buf.trim_end().to_string();
if !line.is_empty() {
batch.push(LokiEntry {
timestamp: chrono::Utc::now(),
line,
});
if batch.len() >= 100 {
if let Some(ref loki) = loki {
if let Err(e) = loki.push(std::mem::take(&mut batch)).await {
tracing::warn!(error = %e, "Loki push failed");
}
}
}
}
}
}
}
}
}

View File

@ -1,101 +0,0 @@
use metrics::{
Counter, Gauge, Histogram, Unit, describe_counter, describe_gauge, describe_histogram,
};
pub fn init() {
describe_gauge!(
"aggregator_targets_total",
Unit::Count,
"Total number of scrape targets known to the aggregator"
);
describe_gauge!(
"aggregator_targets_healthy",
Unit::Count,
"Number of scrape targets that responded last scrape"
);
describe_counter!(
"aggregator_scrape_total",
Unit::Count,
"Total number of scrape attempts"
);
describe_counter!(
"aggregator_scrape_success",
Unit::Count,
"Successful scrapes"
);
describe_counter!(
"aggregator_scrape_failures",
Unit::Count,
"Failed scrape attempts"
);
describe_counter!(
"aggregator_scrape_errors_parse",
Unit::Count,
"Scrape failures due to parse errors"
);
describe_counter!(
"aggregator_scrape_errors_timeout",
Unit::Count,
"Scrape failures due to timeout"
);
describe_counter!(
"aggregator_scrape_errors_connection",
Unit::Count,
"Scrape failures due to connection errors"
);
describe_counter!(
"aggregator_targets_discovered",
Unit::Count,
"Total targets discovered"
);
describe_counter!(
"aggregator_targets_lost",
Unit::Count,
"Total targets that disappeared"
);
describe_histogram!(
"aggregator_scrape_duration_ms",
Unit::Milliseconds,
"Scrape duration in milliseconds"
);
}
#[derive(Clone)]
#[allow(dead_code)]
pub struct AggMetrics {
pub targets_total: Gauge,
pub targets_healthy: Gauge,
pub scrape_total: Counter,
pub scrape_success: Counter,
pub scrape_failures: Counter,
pub scrape_errors_parse: Counter,
pub scrape_errors_timeout: Counter,
pub scrape_errors_connection: Counter,
pub targets_discovered: Counter,
pub targets_lost: Counter,
pub scrape_duration: Histogram,
}
impl Default for AggMetrics {
fn default() -> Self {
Self {
targets_total: metrics::gauge!("aggregator_targets_total"),
targets_healthy: metrics::gauge!("aggregator_targets_healthy"),
scrape_total: metrics::counter!("aggregator_scrape_total"),
scrape_success: metrics::counter!("aggregator_scrape_success"),
scrape_failures: metrics::counter!("aggregator_scrape_failures"),
scrape_errors_parse: metrics::counter!("aggregator_scrape_errors_parse"),
scrape_errors_timeout: metrics::counter!("aggregator_scrape_errors_timeout"),
scrape_errors_connection: metrics::counter!("aggregator_scrape_errors_connection"),
targets_discovered: metrics::counter!("aggregator_targets_discovered"),
targets_lost: metrics::counter!("aggregator_targets_lost"),
scrape_duration: metrics::histogram!("aggregator_scrape_duration_ms"),
}
}
}
impl AggMetrics {
pub fn new() -> Self {
Self::default()
}
}

View File

@ -1,42 +0,0 @@
use anyhow::Context;
use opentelemetry::trace::TracerProvider;
use opentelemetry_otlp::{SpanExporter, WithExportConfig};
use opentelemetry_sdk::trace as sdktrace;
use tracing_opentelemetry::layer;
use tracing_subscriber::prelude::*;
pub struct OtelGuard {
provider: sdktrace::SdkTracerProvider,
}
impl OtelGuard {
pub async fn shutdown(self) {
if let Err(e) = self.provider.shutdown() {
tracing::warn!(error = %e, "OTLP shutdown error");
}
}
}
pub fn init_otel(endpoint: &str, service_name: &str) -> anyhow::Result<OtelGuard> {
let exporter = SpanExporter::builder()
.with_http()
.with_endpoint(endpoint)
.build()
.context("build OTLP exporter")?;
let tracer_provider = sdktrace::SdkTracerProvider::builder()
.with_batch_exporter(exporter)
.build();
let tracer = tracer_provider.tracer(service_name.to_string());
let otel_layer = layer().with_tracer(tracer);
tracing_subscriber::registry()
.with(otel_layer)
.try_init()
.context("install OTLP tracing subscriber")?;
Ok(OtelGuard {
provider: tracer_provider,
})
}

View File

@ -1,135 +0,0 @@
use awc::Client;
use std::collections::HashMap;
use crate::target::ScrapeTarget;
#[derive(Clone)]
pub struct HttpClient {
client: Client,
}
impl HttpClient {
pub fn new(timeout_secs: u64) -> Self {
let client = Client::builder()
.timeout(std::time::Duration::from_secs(timeout_secs))
.finish();
Self { client }
}
pub async fn scrape(&self, target: &ScrapeTarget) -> ScrapeResult {
let start = std::time::Instant::now();
let url = target.url();
let mut resp = match self.client.get(url).send().await {
Ok(resp) => resp,
Err(e) => {
let msg = e.to_string();
if msg.contains("timeout") || msg.contains("TimedOut") || msg.contains("timed out")
{
return ScrapeResult::Timeout;
}
return ScrapeResult::ConnectionError(msg);
}
};
if !resp.status().is_success() {
return ScrapeResult::HttpError(resp.status().as_u16());
}
let body = match resp.body().await {
Ok(bytes) => String::from_utf8_lossy(&bytes).into_owned(),
Err(e) => return ScrapeResult::ConnectionError(e.to_string()),
};
let scrape_ms = start.elapsed().as_millis() as f64;
ScrapeResult::Success(body, scrape_ms)
}
}
pub enum ScrapeResult {
Success(String, f64),
Timeout,
ConnectionError(String),
HttpError(u16),
}
#[derive(Clone, Debug)]
pub struct PromMetric {
pub name: String,
pub value: f64,
pub labels: HashMap<String, String>,
}
pub fn parse_prometheus(body: &str) -> Vec<PromMetric> {
let mut metrics = Vec::new();
for line in body.lines() {
let line = line.trim();
if line.is_empty() || line.starts_with('#') {
continue;
}
let (name_and_labels, value_str) = match line.find(' ') {
Some(pos) => (&line[..pos], &line[pos + 1..]),
None => continue,
};
let value: f64 = match value_str
.split_whitespace()
.next()
.and_then(|v| v.parse().ok())
{
Some(v) => v,
None => continue,
};
let (metric_name, labels) = if let Some(brace) = name_and_labels.find('{') {
let name = &name_and_labels[..brace];
let label_str = &name_and_labels[brace + 1..name_and_labels.len() - 1];
let labels = parse_labels(label_str);
(name.to_string(), labels)
} else {
(name_and_labels.to_string(), HashMap::new())
};
metrics.push(PromMetric {
name: metric_name,
value,
labels,
});
}
metrics
}
pub fn parse_labels(s: &str) -> HashMap<String, String> {
let mut labels = HashMap::new();
let mut remaining = s;
while !remaining.is_empty() {
if let Some(eq) = remaining.find('=') {
let key = remaining[..eq].trim().to_string();
remaining = &remaining[eq + 1..];
let (value, rest) = if remaining.starts_with('"') {
let end = remaining[1..]
.find('"')
.map(|p| p + 1)
.unwrap_or(remaining.len());
(&remaining[1..end], &remaining[end + 1..])
} else if remaining.starts_with('\'') {
let end = remaining[1..]
.find('\'')
.map(|p| p + 1)
.unwrap_or(remaining.len());
(&remaining[1..end], &remaining[end + 1..])
} else {
let end = remaining
.find(|c: char| !c.is_alphanumeric() && c != '_' && c != '-')
.unwrap_or(remaining.len());
(&remaining[..end], &remaining[end..])
};
labels.insert(key, value.to_string());
remaining = rest.trim_start_matches(',').trim_start();
} else {
break;
}
}
labels
}

View File

@ -1,217 +0,0 @@
//! Stats store: receives expanded push payloads from all apps,
//! aggregates over time, computes derived statistics (p99 etc),
//! and provides JSON API for external consumption.
use serde::Serialize;
use std::collections::HashMap;
use std::sync::Arc;
use tokio::sync::RwLock;
/// Per-app, per-instance aggregated stats entry.
#[derive(Debug, Clone, Default, Serialize)]
pub struct AppStats {
/// Last seen timestamp.
pub last_seen: i64,
/// Number of push samples received.
pub sample_count: u64,
// ── HTTP ─────────────────────────────────────────────────────
pub requests_total: u64,
pub request_duration_ms_total: u64,
pub requests_2xx: u64,
pub requests_4xx: u64,
pub requests_5xx: u64,
pub endpoints: HashMap<String, u64>,
// ── System ───────────────────────────────────────────────────
pub cpu_usage_percent: f32,
pub memory_used_mb: u64,
pub memory_total_mb: u64,
pub uptime_secs: u64,
// ── Business counters ────────────────────────────────────────
pub business: HashMap<String, f64>,
// ── Token usage ──────────────────────────────────────────────
pub ai_input_tokens_total: i64,
pub ai_output_tokens_total: i64,
pub ai_calls_total: i64,
pub ai_calls_success: i64,
pub ai_calls_failure: i64,
pub token_by_model: HashMap<String, ModelTokenStats>,
// ── Tasks ────────────────────────────────────────────────────
pub tasks_queued: i64,
pub tasks_running: i64,
pub tasks_completed: i64,
pub tasks_failed: i64,
// ── Latency ──────────────────────────────────────────────────
pub latency: HashMap<String, LatencyStats>,
// ── Logs ─────────────────────────────────────────────────────
#[serde(skip_serializing)]
pub logs: Vec<(i64, String)>,
}
#[derive(Debug, Clone, Default, Serialize)]
pub struct ModelTokenStats {
pub input_tokens: i64,
pub output_tokens: i64,
pub calls: i64,
}
#[derive(Debug, Clone, Default, Serialize)]
pub struct LatencyStats {
pub p50_ms: f64,
pub p90_ms: f64,
pub p99_ms: f64,
pub max_ms: f64,
pub count: u64,
}
/// The global stats store: app_name → AppStats.
pub type StatsStore = Arc<RwLock<HashMap<String, AppStats>>>;
/// Merge a new push payload into the stats store.
pub async fn merge_push_payload(
store: &StatsStore,
app: &str,
_instance: &str,
timestamp: i64,
http: Option<&observability::push::HttpPayload>,
system: Option<&observability::push::SystemPayload>,
business: &HashMap<String, f64>,
token_usage: Option<&observability::push::TokenUsagePayload>,
tasks: Option<&observability::push::TaskStatsPayload>,
latency: &HashMap<String, observability::push::LatencySnapshot>,
logs: &[observability::push::LogEntry],
) {
// Use app_name as key (merge across instances for aggregation)
let mut guard = store.write().await;
let entry = guard.entry(app.to_string()).or_default();
entry.last_seen = timestamp;
entry.sample_count += 1;
// HTTP — accumulate (not replace, so we get totals over time)
if let Some(http) = http {
entry.requests_total = http.requests_total;
entry.request_duration_ms_total = http.request_duration_ms_total;
entry.requests_2xx = http.requests_2xx;
entry.requests_4xx = http.requests_4xx;
entry.requests_5xx = http.requests_5xx;
for (ep, count) in &http.endpoints {
*entry.endpoints.entry(ep.clone()).or_insert(0) = *count;
}
}
// System — replace (current snapshot, not cumulative)
if let Some(sys) = system {
entry.cpu_usage_percent = sys.cpu_usage_percent;
entry.memory_used_mb = sys.memory_used_mb;
entry.memory_total_mb = sys.memory_total_mb;
entry.uptime_secs = sys.uptime_secs;
}
// Business — replace with latest snapshot
entry.business = business.clone();
// Token usage — replace with latest
if let Some(tu) = token_usage {
entry.ai_input_tokens_total = tu.ai_input_tokens_total;
entry.ai_output_tokens_total = tu.ai_output_tokens_total;
entry.ai_calls_total = tu.ai_calls_total;
entry.ai_calls_success = tu.ai_calls_success;
entry.ai_calls_failure = tu.ai_calls_failure;
for (model, usage) in &tu.by_model {
let ms = entry.token_by_model.entry(model.clone()).or_default();
ms.input_tokens = usage.input_tokens;
ms.output_tokens = usage.output_tokens;
ms.calls = usage.calls;
}
}
// Tasks — replace with latest
if let Some(t) = tasks {
entry.tasks_queued = t.queued;
entry.tasks_running = t.running;
entry.tasks_completed = t.completed;
entry.tasks_failed = t.failed;
}
// Latency — replace with latest snapshots
for (endpoint, snap) in latency {
let ls = entry.latency.entry(endpoint.clone()).or_default();
ls.p50_ms = snap.p50_ms;
ls.p90_ms = snap.p90_ms;
ls.p99_ms = snap.p99_ms;
ls.max_ms = snap.max_ms;
ls.count = snap.count;
}
// Logs — append (keep last 300 lines)
for log in logs {
entry.logs.push((
log.timestamp,
format!("[{}] {}", log.level.to_lowercase(), log.message),
));
}
let cutoff = chrono::Utc::now().timestamp() - 300;
entry.logs.retain(|(ts, _)| *ts >= cutoff);
}
/// Dashboard response combining all apps' stats.
#[derive(Debug, Serialize)]
pub struct DashboardResponse {
/// Timestamp of this snapshot.
pub timestamp: i64,
/// Total number of app instances reporting.
pub app_count: u64,
/// Per-app aggregated stats.
pub apps: HashMap<String, AppStats>,
/// Derived: average p99 latency across all apps.
pub avg_p99_ms: f64,
/// Derived: total tokens consumed across all apps.
pub total_input_tokens: i64,
pub total_output_tokens: i64,
/// Derived: total AI calls across all apps.
pub total_ai_calls: i64,
}
/// Build the dashboard response from the stats store.
pub async fn build_dashboard(store: &StatsStore) -> DashboardResponse {
let guard = store.read().await;
let mut avg_p99 = 0.0;
let mut p99_count = 0;
let mut total_input = 0i64;
let mut total_output = 0i64;
let mut total_calls = 0i64;
for (_, stats) in guard.iter() {
total_input += stats.ai_input_tokens_total;
total_output += stats.ai_output_tokens_total;
total_calls += stats.ai_calls_total;
for (_, lat) in &stats.latency {
avg_p99 += lat.p99_ms;
p99_count += 1;
}
}
let avg_p99_ms = if p99_count > 0 {
avg_p99 / p99_count as f64
} else {
0.0
};
DashboardResponse {
timestamp: chrono::Utc::now().timestamp(),
app_count: guard.len() as u64,
apps: guard.clone(),
avg_p99_ms,
total_input_tokens: total_input,
total_output_tokens: total_output,
total_ai_calls: total_calls,
}
}

View File

@ -1,36 +0,0 @@
use anyhow::Context;
use serde::{Deserialize, Serialize};
use std::collections::HashMap;
#[derive(Clone, Debug, Serialize, Deserialize)]
pub struct ScrapeTarget {
pub name: String,
pub addr: String,
#[serde(default = "default_metrics_path")]
pub metrics_path: String,
#[serde(default)]
pub labels: HashMap<String, String>,
}
fn default_metrics_path() -> String {
"/metrics".to_string()
}
impl ScrapeTarget {
pub fn url(&self) -> String {
if self.metrics_path.starts_with("http") {
self.metrics_path.clone()
} else {
format!("http://{}{}", self.addr, self.metrics_path)
}
}
}
pub async fn load_targets_from_file(path: &str) -> anyhow::Result<Vec<ScrapeTarget>> {
let content = tokio::fs::read_to_string(path)
.await
.context("read targets file")?;
let targets: Vec<ScrapeTarget> =
serde_json::from_str(&content).with_context(|| format!("parse targets file {path}"))?;
Ok(targets)
}

View File

@ -1,13 +0,0 @@
[package]
name = "migrate-cli"
version.workspace = true
edition.workspace = true
[dependencies]
migrate.workspace = true
sea-orm = { workspace = true, features = ["sqlx-all", "runtime-tokio"] }
tokio = { workspace = true, features = ["rt-multi-thread", "macros"] }
anyhow.workspace = true
clap.workspace = true
dotenvy.workspace = true
config = { workspace = true }

View File

@ -1,102 +0,0 @@
use anyhow::Context;
use clap::Command;
use migrate::MigratorTrait;
use sea_orm::{Database, DatabaseConnection};
#[tokio::main]
async fn main() -> anyhow::Result<()> {
dotenvy::dotenv().ok();
config::AppConfig::load();
let cmd = Command::new("migrate")
.about("Database migration CLI")
.arg(
clap::Arg::new("steps")
.help("Number of migrations (for up/down)")
.required(false)
.index(1),
)
.subcommand(Command::new("up").about("Apply pending migrations"))
.subcommand(Command::new("down").about("Revert applied migrations"))
.subcommand(Command::new("fresh").about("Drop all tables and re-apply"))
.subcommand(Command::new("refresh").about("Revert all then re-apply"))
.subcommand(Command::new("reset").about("Revert all applied migrations"))
.subcommand(Command::new("status").about("Show migration status"))
.try_get_matches()
.map_err(|e| anyhow::anyhow!("{}", e))?;
let db_url = config::AppConfig::load().database_url()?;
let db: DatabaseConnection = Database::connect(&db_url).await?;
match cmd.subcommand_name() {
Some("up") => {
let steps = cmd
.get_one::<String>("steps")
.and_then(|s| s.parse().ok())
.unwrap_or(0);
run_up(&db, steps).await?;
}
Some("down") => {
let steps = cmd
.get_one::<String>("steps")
.and_then(|s| s.parse().ok())
.unwrap_or(1);
run_down(&db, steps).await?;
}
Some("fresh") => run_fresh(&db).await?,
Some("refresh") => run_refresh(&db).await?,
Some("reset") => run_reset(&db).await?,
Some("status") => run_status(&db).await?,
_ => {
eprintln!(
"Usage: migrate <command>\nCommands: up, down, fresh, refresh, reset, status"
);
std::process::exit(1);
}
}
Ok(())
}
async fn run_up(db: &DatabaseConnection, steps: u32) -> anyhow::Result<()> {
migrate::Migrator::up(db, if steps == 0 { None } else { Some(steps) })
.await
.context("failed to run migrations up")?;
Ok(())
}
async fn run_down(db: &DatabaseConnection, steps: u32) -> anyhow::Result<()> {
migrate::Migrator::down(db, Some(steps))
.await
.context("failed to run migrations down")?;
Ok(())
}
async fn run_fresh(db: &DatabaseConnection) -> anyhow::Result<()> {
migrate::Migrator::fresh(db)
.await
.context("failed to run migrations fresh")?;
Ok(())
}
async fn run_refresh(db: &DatabaseConnection) -> anyhow::Result<()> {
migrate::Migrator::refresh(db)
.await
.context("failed to run migrations refresh")?;
Ok(())
}
async fn run_reset(db: &DatabaseConnection) -> anyhow::Result<()> {
migrate::Migrator::reset(db)
.await
.context("failed to run migrations reset")?;
Ok(())
}
async fn run_status(db: &DatabaseConnection) -> anyhow::Result<()> {
migrate::Migrator::status(db)
.await
.context("failed to get migration status")?;
Ok(())
}

View File

@ -1,21 +0,0 @@
[package]
name = "static-server"
version.workspace = true
edition.workspace = true
[dependencies]
actix-web = { workspace = true }
actix-files = { workspace = true }
actix-cors = { workspace = true }
observability = { workspace = true }
metrics-exporter-prometheus = "0.13"
tokio = { workspace = true, features = ["full"] }
futures = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
mime = { workspace = true }
mime_guess2 = { workspace = true }
slog = { workspace = true }
anyhow = { workspace = true }
env_logger = { workspace = true }
log = "0.4"

View File

@ -1,212 +0,0 @@
use actix_cors::Cors;
use actix_files::Files;
use actix_web::dev::{Service, ServiceRequest, ServiceResponse};
use actix_web::{App, HttpResponse, HttpServer, http::header, web};
use futures::future::LocalBoxFuture;
use log::info;
use observability::{HttpMetrics, init_tracing_subscriber, install_recorder, push::MetricsPusher};
use std::path::PathBuf;
use std::sync::Arc;
use std::task::{Context, Poll};
use std::time::Instant;
/// Static file server for avatar, blob, and other static files
/// Serves files from /data/{type} directories
#[derive(Clone)]
struct StaticConfig {
root: PathBuf,
cors_enabled: bool,
}
impl StaticConfig {
fn from_env() -> Self {
let root = std::env::var("STATIC_ROOT").unwrap_or_else(|_| "/data".to_string());
let cors = std::env::var("STATIC_CORS").unwrap_or_else(|_| "true".to_string());
Self {
root: PathBuf::from(root),
cors_enabled: cors == "true" || cors == "1",
}
}
fn ensure_dir(&self, name: &str) -> PathBuf {
let dir = self.root.join(name);
if !dir.exists() {
std::fs::create_dir_all(&dir).ok();
}
dir
}
}
async fn health() -> HttpResponse {
HttpResponse::Ok().json(serde_json::json!({
"status": "ok",
"service": "static-server"
}))
}
/// Custom middleware that logs requests except for noisy paths (health, metrics, static files).
struct RequestLogger;
impl<S, B> actix_web::dev::Transform<S, ServiceRequest> for RequestLogger
where
S: Service<ServiceRequest, Response = ServiceResponse<B>, Error = actix_web::Error>,
S::Future: 'static,
B: 'static,
{
type Response = ServiceResponse<B>;
type Error = actix_web::Error;
type Transform = RequestLoggerService<S>;
type InitError = ();
type Future = futures::future::Ready<Result<Self::Transform, Self::InitError>>;
fn new_transform(&self, service: S) -> Self::Future {
futures::future::ok(RequestLoggerService {
service,
_marker: std::marker::PhantomData,
})
}
}
struct RequestLoggerService<S> {
service: S,
_marker: std::marker::PhantomData<fn(ServiceRequest)>,
}
impl<S, B> Service<ServiceRequest> for RequestLoggerService<S>
where
S: Service<ServiceRequest, Response = ServiceResponse<B>, Error = actix_web::Error>,
S::Future: 'static,
B: 'static,
{
type Response = ServiceResponse<B>;
type Error = actix_web::Error;
type Future = LocalBoxFuture<'static, Result<Self::Response, Self::Error>>;
fn poll_ready(&self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
self.service.poll_ready(cx)
}
fn call(&self, req: ServiceRequest) -> Self::Future {
let path = req.path().to_string();
let method = req.method().to_string();
let should_log = !(path == "/health"
|| path == "/metrics"
|| path.starts_with("/ws")
|| path.starts_with("/avatar")
|| path.starts_with("/blob")
|| path.starts_with("/media")
|| path.starts_with("/static"));
let start = Instant::now();
let fut = self.service.call(req);
Box::pin(async move {
let res = fut.await?;
if should_log {
info!(
target: "static_server",
"{} {} {} {:?}",
method,
path,
res.status().as_u16(),
start.elapsed()
);
}
Ok(res)
})
}
}
#[actix_web::main]
async fn main() -> anyhow::Result<()> {
init_tracing_subscriber("info", false);
let prometheus_handle = Arc::new(install_recorder());
let http_metrics = Arc::new(HttpMetrics::new());
// Metrics pusher: periodically push all metrics to apps/metrics aggregator
if let Some(push_url) = std::env::var("METRICS_PUSH_URL").ok() {
let pusher = MetricsPusher::new(&push_url, "static");
pusher.spawn(
http_metrics.clone(),
prometheus_handle.clone(),
std::time::Duration::from_secs(15),
);
info!("Metrics pusher started (interval 15s, url: {})", push_url);
}
let cfg = StaticConfig::from_env();
let bind = std::env::var("STATIC_BIND").unwrap_or_else(|_| "0.0.0.0:8081".to_string());
println!("Static file server starting...");
println!(" Root: {:?}", cfg.root);
println!(" Bind: {}", bind);
println!(
" CORS: {}",
if cfg.cors_enabled {
"enabled"
} else {
"disabled"
}
);
// Ensure all directories exist
for name in ["avatar", "blob", "media", "static"] {
let dir = cfg.ensure_dir(name);
println!(" {} dir: {:?}", name, dir);
}
let root = cfg.root.clone();
let cors_enabled = cfg.cors_enabled;
HttpServer::new(move || {
let root = root.clone();
let cors = if cors_enabled {
// WARNING: allow_any_origin is intentional for static asset serving (CDN mode)
// Ensure no sensitive files are served from this directory
Cors::default()
.allow_any_origin()
.allowed_methods(vec!["GET", "HEAD", "OPTIONS"])
.allowed_headers(vec![
header::AUTHORIZATION,
header::ACCEPT,
header::CONTENT_TYPE,
])
.max_age(3600)
} else {
Cors::permissive()
};
App::new()
.wrap(cors)
.wrap(RequestLogger)
.route("/health", web::get().to(health))
.service(
Files::new("/avatar", root.join("avatar"))
.prefer_utf8(true)
.index_file("index.html"),
)
.service(
Files::new("/blob", root.join("blob"))
.prefer_utf8(true)
.index_file("index.html"),
)
.service(
Files::new("/media", root.join("media"))
.prefer_utf8(true)
.index_file("index.html"),
)
.service(
Files::new("/static", root.join("static"))
.prefer_utf8(true)
.index_file("index.html"),
)
})
.bind(&bind)?
.run()
.await?;
Ok(())
}

View File

@ -1,88 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
# ── helpers ──────────────────────────────────────────────────────────
RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'; NC='\033[0m'
log() { echo -e "${GREEN}[OK]${NC} $*"; }
warn() { echo -e "${YELLOW}[WARN]${NC} $*"; }
err() { echo -e "${RED}[ERR]${NC} $*"; exit 1; }
command_exists() { command -v "$1" &>/dev/null; }
# ── 1. Rust ─────────────────────────────────────────────────────────
if command_exists rustc; then
log "Rust $(rustc --version)"
else
warn "Rust not found, installing via rustup..."
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
# shellcheck disable=SC1091
source "$HOME/.cargo/env"
log "Rust installed: $(rustc --version)"
fi
# ── 2. Node.js ──────────────────────────────────────────────────────
if command_exists node; then
log "Node.js $(node --version)"
else
warn "Node.js not found, installing via nvm..."
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash
# shellcheck disable=SC1090
export NVM_DIR="${HOME}/.nvm"
[ -s "$NVM_DIR/nvm.sh" ] && source "$NVM_DIR/nvm.sh"
nvm install --lts
log "Node.js installed: $(node --version)"
fi
# ── 2b. Bun ─────────────────────────────────────────────────────────
if command_exists bun; then
log "Bun $(bun --version)"
else
warn "Bun not found, installing..."
curl -fsSL https://bun.sh/install | bash
# shellcheck disable=SC1091
[ -s "$HOME/.bun/_bun" ] && export PATH="$HOME/.bun/bin:$PATH"
log "Bun installed: $(bun --version)"
fi
# ── 3. Docker ───────────────────────────────────────────────────────
if command_exists docker; then
log "Docker $(docker --version)"
else
warn "Docker not found, installing..."
curl -fsSL https://get.docker.com | sh
log "Docker installed: $(docker --version)"
fi
# ── 4. Frontend build ───────────────────────────────────────────────
log "Running bun install..."
bun install
log "Running bun run build..."
bun run build
# ── 5. Rust build ───────────────────────────────────────────────────
log "Running cargo build --release --workspace..."
cargo build --release --workspace
# ── 6. Docker images ────────────────────────────────────────────────
TAG=$(git rev-parse --short HEAD)
log "Building Docker images with tag: $TAG"
IMAGES=(
"docker/app.Dockerfile app:$TAG"
"docker/email.Dockerfile email-worker:$TAG"
"docker/githook.Dockerfile git-hook:$TAG"
"docker/gitserver.Dockerfile gitserver:$TAG"
"docker/metrics.Dockerfile metrics-aggregator:$TAG"
"docker/static.Dockerfile static-server:$TAG"
"docker/gingress.Dockerfile gingress:$TAG"
)
for entry in "${IMAGES[@]}"; do
read -r dockerfile tag <<< "$entry"
log "Building $tag..."
docker build -f "$dockerfile" -t "$tag" .
done
log "All images built successfully."
docker images | grep -E "app|email-worker|git-hook|gitserver|metrics-aggregator|static-server|gingress" | grep "$TAG" || true

1813
bun.lock

File diff suppressed because it is too large Load Diff

0
chart/app/.helmignore Normal file
View File

17
chart/app/Chart.yaml Normal file
View File

@ -0,0 +1,17 @@
apiVersion: v2
name: gitdataai
description: GitDataAI — Git platform with AI-powered code agents
type: application
version: 0.2.0
appVersion: "1.0.0"
keywords:
- git
- ai
- agent
- code-review
- cicd
maintainers:
- name: gitdataai-team

View File

@ -0,0 +1,11 @@
{{- $svcNames := list "gitdata" "gitpod" "gitsync" "email" }}
{{- range $svcName := $svcNames }}
{{- $svcCfg := index $.Values $svcName }}
{{- if and $svcCfg.enabled (not (eq $svcName "email")) }}
{{- if eq $svcName "gitpod" }}
{{- printf " http://%s-http.%s.svc.cluster.local:%d/" (include "app.serviceFullname" (dict "root" $ "name" $svcName)) (include "app.namespace" $) (int $svcCfg.service.port) }}
{{- else }}
{{- printf " http://%s.%s.svc.cluster.local:%d/" (include "app.serviceFullname" (dict "root" $ "name" $svcName)) (include "app.namespace" $) (int $svcCfg.service.port) }}
{{- end }}
{{- end }}
{{- end }}

View File

@ -0,0 +1,125 @@
{{/*
Expand the name of the chart.
*/}}
{{- define "app.name" -}}
{{- default .Chart.Name .Values.global.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Create a default fully qualified app name.
*/}}
{{- define "app.fullname" -}}
{{- if .Values.global.fullnameOverride }}
{{- .Values.global.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.global.nameOverride }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "app.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Common labels.
*/}}
{{- define "app.labels" -}}
helm.sh/chart: {{ include "app.chart" . }}
app.kubernetes.io/name: {{ include "app.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}
{{/*
Service labels.
*/}}
{{- define "app.serviceLabels" -}}
{{- $root := .root }}
{{- $name := .name }}
helm.sh/chart: {{ include "app.chart" $root }}
app.kubernetes.io/name: {{ $name }}
app.kubernetes.io/instance: {{ $root.Release.Name }}
app.kubernetes.io/component: {{ $name }}
{{- if $root.Chart.AppVersion }}
app.kubernetes.io/version: {{ $root.Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ $root.Release.Service }}
{{- end }}
{{/*
Selector labels.
*/}}
{{- define "app.serviceSelectorLabels" -}}
{{- $root := .root }}
{{- $name := .name }}
app.kubernetes.io/name: {{ $name }}
app.kubernetes.io/instance: {{ $root.Release.Name }}
app.kubernetes.io/component: {{ $name }}
{{- end }}
{{/*
Fully qualified service name: <release>-<chart>-<serviceName>
*/}}
{{- define "app.serviceFullname" -}}
{{- $root := .root }}
{{- $name := .name }}
{{- if $root.Values.global.fullnameOverride }}
{{- printf "%s-%s" $root.Values.global.fullnameOverride $name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $chartName := default $root.Chart.Name $root.Values.global.nameOverride }}
{{- printf "%s-%s-%s" $root.Release.Name $chartName $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{/*
Namespace.
*/}}
{{- define "app.namespace" -}}
{{- .Values.global.namespace | default .Release.Namespace }}
{{- end }}
{{/*
ServiceAccount name.
*/}}
{{- define "app.serviceAccountName" -}}
{{- if .Values.serviceAccount.name }}
{{- .Values.serviceAccount.name }}
{{- else }}
{{- include "app.fullname" . }}
{{- end }}
{{- end }}
{{/*
Docker image reference.
*/}}
{{- define "app.image" -}}
{{- $globalRegistry := .root.Values.global.image.registry }}
{{- $registry := .svc.registry | default $globalRegistry }}
{{- $name := .svc.name }}
{{- $tag := .svc.tag | default .root.Values.global.image.tag | default "latest" }}
{{- printf "%s/%s:%s" $registry $name $tag }}
{{- end }}
{{/*
Image pull secrets.
*/}}
{{- define "app.imagePullSecrets" -}}
{{- with .Values.global.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 2 }}
{{- end }}
{{- end }}
{{/*
Gitpod RPC cluster DNS address.
*/}}
{{- define "app.gitpodRpcAddr" -}}
{{ include "app.serviceFullname" (dict "root" . "name" "gitpod") }}-rpc.{{ include "app.namespace" . }}
{{- end }}

View File

@ -0,0 +1,28 @@
{{/*
Single shared ConfigMap for all services.
Merges global.env with service-specific overrides.
*/}}
{{- $allEnv := deepCopy ($.Values.global.env | default dict) }}
{{- /* Auto-fill APP_GIT_RPC_ADDR for gitdata -> gitpod-rpc service */}}
{{- if and $.Values.gitdata.enabled (not $.Values.gitdata.env.APP_GIT_RPC_ADDR) }}
{{- $_ := set $allEnv "APP_GIT_RPC_ADDR" (include "app.gitpodRpcAddr" $) }}
{{- end }}
{{- range $svcName, $svc := dict "gitdata" $.Values.gitdata "gitpod" $.Values.gitpod "gitsync" $.Values.gitsync "email" $.Values.email }}
{{- if $svc.enabled }}
{{- $allEnv = merge $allEnv ($svc.env | default dict) }}
{{- end }}
{{- end }}
{{- if $allEnv }}
---
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ include "app.fullname" $ }}
namespace: {{ $.Values.global.namespace | default $.Release.Namespace }}
labels:
{{- include "app.labels" $ | nindent 4 }}
data:
{{- range $k, $v := $allEnv }}
{{ $k }}: {{ $v | quote }}
{{- end }}
{{- end }}

View File

@ -0,0 +1,381 @@
{{/*
Deployments — One per enabled service.
All pods share app-data-pvc mounted at /data.
*/}}
{{/* ============================================================
gitdata — Main API server
============================================================ */}}
{{- if .Values.gitdata.enabled }}
{{- $svc := .Values.gitdata }}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "app.serviceFullname" (dict "root" . "name" "gitdata") }}
namespace: {{ include "app.namespace" . }}
labels:
{{- include "app.serviceLabels" (dict "root" . "name" "gitdata") | nindent 4 }}
spec:
replicas: {{ $svc.replicaCount }}
selector:
matchLabels:
{{- include "app.serviceSelectorLabels" (dict "root" . "name" "gitdata") | nindent 6 }}
template:
metadata:
annotations:
{{- with $svc.podAnnotations }}
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "app.serviceLabels" (dict "root" . "name" "gitdata") | nindent 8 }}
spec:
{{- include "app.imagePullSecrets" . | nindent 6 }}
serviceAccountName: {{ include "app.serviceAccountName" . }}
securityContext:
{{- toYaml $svc.podSecurityContext | nindent 8 }}
containers:
- name: {{ .Chart.Name }}
image: {{ include "app.image" (dict "root" . "svc" $svc.image) }}
imagePullPolicy: {{ .Values.global.image.pullPolicy }}
ports:
- name: http
containerPort: 8080
protocol: TCP
envFrom:
- configMapRef:
name: {{ include "app.fullname" . }}
resources:
{{- toYaml $svc.resources | nindent 12 }}
securityContext:
{{- toYaml $svc.securityContext | nindent 12 }}
volumeMounts:
- name: data
mountPath: /data
{{- with $svc.volumeMounts }}
{{- toYaml . | nindent 12 }}
{{- end }}
startupProbe:
httpGet:
path: {{ $svc.startupProbe.httpGet.path }}
port: {{ $svc.startupProbe.httpGet.port }}
initialDelaySeconds: {{ $svc.startupProbe.initialDelaySeconds }}
periodSeconds: {{ $svc.startupProbe.periodSeconds }}
failureThreshold: {{ $svc.startupProbe.failureThreshold }}
livenessProbe:
httpGet:
path: {{ $svc.livenessProbe.httpGet.path }}
port: {{ $svc.livenessProbe.httpGet.port }}
periodSeconds: {{ $svc.livenessProbe.periodSeconds }}
readinessProbe:
httpGet:
path: {{ $svc.readinessProbe.httpGet.path }}
port: {{ $svc.readinessProbe.httpGet.port }}
periodSeconds: {{ $svc.readinessProbe.periodSeconds }}
volumes:
- name: data
persistentVolumeClaim:
claimName: app-data-pvc
{{- with $svc.volumes }}
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with $svc.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with $svc.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with $svc.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}
{{/* ============================================================
gitpod — Git protocol server (HTTP + SSH + gRPC)
============================================================ */}}
{{- if .Values.gitpod.enabled }}
{{- $svc := .Values.gitpod }}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "app.serviceFullname" (dict "root" . "name" "gitpod") }}
namespace: {{ include "app.namespace" . }}
labels:
{{- include "app.serviceLabels" (dict "root" . "name" "gitpod") | nindent 4 }}
spec:
replicas: {{ $svc.replicaCount }}
selector:
matchLabels:
{{- include "app.serviceSelectorLabels" (dict "root" . "name" "gitpod") | nindent 6 }}
template:
metadata:
annotations:
{{- with $svc.podAnnotations }}
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "app.serviceLabels" (dict "root" . "name" "gitpod") | nindent 8 }}
spec:
{{- include "app.imagePullSecrets" . | nindent 6 }}
serviceAccountName: {{ include "app.serviceAccountName" . }}
securityContext:
{{- toYaml $svc.podSecurityContext | nindent 8 }}
containers:
- name: {{ .Chart.Name }}
image: {{ include "app.image" (dict "root" . "svc" $svc.image) }}
imagePullPolicy: {{ .Values.global.image.pullPolicy }}
ports:
- name: http
containerPort: 8080
protocol: TCP
- name: ssh
containerPort: 2222
protocol: TCP
- name: grpc
containerPort: 50051
protocol: TCP
envFrom:
- configMapRef:
name: {{ include "app.fullname" . }}
resources:
{{- toYaml $svc.resources | nindent 12 }}
securityContext:
{{- toYaml $svc.securityContext | nindent 12 }}
volumeMounts:
- name: data
mountPath: /data
{{- with $svc.volumeMounts }}
{{- toYaml . | nindent 12 }}
{{- end }}
{{- if $svc.sshHostKeySecret }}
- name: ssh-host-key
mountPath: /etc/ssh
readOnly: true
{{- end }}
startupProbe:
httpGet:
path: {{ $svc.startupProbe.httpGet.path }}
port: {{ $svc.startupProbe.httpGet.port }}
initialDelaySeconds: {{ $svc.startupProbe.initialDelaySeconds }}
periodSeconds: {{ $svc.startupProbe.periodSeconds }}
failureThreshold: {{ $svc.startupProbe.failureThreshold }}
livenessProbe:
httpGet:
path: {{ $svc.livenessProbe.httpGet.path }}
port: {{ $svc.livenessProbe.httpGet.port }}
periodSeconds: {{ $svc.livenessProbe.periodSeconds }}
readinessProbe:
httpGet:
path: {{ $svc.readinessProbe.httpGet.path }}
port: {{ $svc.readinessProbe.httpGet.port }}
periodSeconds: {{ $svc.readinessProbe.periodSeconds }}
volumes:
- name: data
persistentVolumeClaim:
claimName: app-data-pvc
{{- if $svc.sshHostKeySecret }}
- name: ssh-host-key
secret:
secretName: {{ $svc.sshHostKeySecret }}
defaultMode: 0600
{{- end }}
{{- with $svc.volumes }}
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with $svc.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with $svc.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with $svc.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}
{{/* ============================================================
gitsync — Git sync worker
============================================================ */}}
{{- if .Values.gitsync.enabled }}
{{- $svc := .Values.gitsync }}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "app.serviceFullname" (dict "root" . "name" "gitsync") }}
namespace: {{ include "app.namespace" . }}
labels:
{{- include "app.serviceLabels" (dict "root" . "name" "gitsync") | nindent 4 }}
spec:
replicas: {{ $svc.replicaCount }}
selector:
matchLabels:
{{- include "app.serviceSelectorLabels" (dict "root" . "name" "gitsync") | nindent 6 }}
template:
metadata:
annotations:
{{- with $svc.podAnnotations }}
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "app.serviceLabels" (dict "root" . "name" "gitsync") | nindent 8 }}
spec:
{{- include "app.imagePullSecrets" . | nindent 6 }}
serviceAccountName: {{ include "app.serviceAccountName" . }}
securityContext:
{{- toYaml $svc.podSecurityContext | nindent 8 }}
containers:
- name: {{ .Chart.Name }}
image: {{ include "app.image" (dict "root" . "svc" $svc.image) }}
imagePullPolicy: {{ .Values.global.image.pullPolicy }}
ports:
- name: health
containerPort: 8081
protocol: TCP
envFrom:
- configMapRef:
name: {{ include "app.fullname" . }}
resources:
{{- toYaml $svc.resources | nindent 12 }}
securityContext:
{{- toYaml $svc.securityContext | nindent 12 }}
volumeMounts:
- name: data
mountPath: /data
{{- with $svc.volumeMounts }}
{{- toYaml . | nindent 12 }}
{{- end }}
startupProbe:
httpGet:
path: {{ $svc.startupProbe.httpGet.path }}
port: {{ $svc.startupProbe.httpGet.port }}
initialDelaySeconds: {{ $svc.startupProbe.initialDelaySeconds }}
periodSeconds: {{ $svc.startupProbe.periodSeconds }}
failureThreshold: {{ $svc.startupProbe.failureThreshold }}
livenessProbe:
httpGet:
path: {{ $svc.livenessProbe.httpGet.path }}
port: {{ $svc.livenessProbe.httpGet.port }}
periodSeconds: {{ $svc.livenessProbe.periodSeconds }}
readinessProbe:
httpGet:
path: {{ $svc.readinessProbe.httpGet.path }}
port: {{ $svc.readinessProbe.httpGet.port }}
periodSeconds: {{ $svc.readinessProbe.periodSeconds }}
volumes:
- name: data
persistentVolumeClaim:
claimName: app-data-pvc
{{- with $svc.volumes }}
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with $svc.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with $svc.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with $svc.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}
{{/* ============================================================
email — Email worker service
============================================================ */}}
{{- if .Values.email.enabled }}
{{- $svc := .Values.email }}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "app.serviceFullname" (dict "root" . "name" "email") }}
namespace: {{ include "app.namespace" . }}
labels:
{{- include "app.serviceLabels" (dict "root" . "name" "email") | nindent 4 }}
spec:
replicas: {{ $svc.replicaCount }}
selector:
matchLabels:
{{- include "app.serviceSelectorLabels" (dict "root" . "name" "email") | nindent 6 }}
template:
metadata:
annotations:
{{- with $svc.podAnnotations }}
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "app.serviceLabels" (dict "root" . "name" "email") | nindent 8 }}
spec:
{{- include "app.imagePullSecrets" . | nindent 6 }}
serviceAccountName: {{ include "app.serviceAccountName" . }}
securityContext:
{{- toYaml $svc.podSecurityContext | nindent 8 }}
containers:
- name: {{ .Chart.Name }}
image: {{ include "app.image" (dict "root" . "svc" $svc.image) }}
imagePullPolicy: {{ .Values.global.image.pullPolicy }}
ports:
- name: health
containerPort: 8083
protocol: TCP
envFrom:
- configMapRef:
name: {{ include "app.fullname" . }}
resources:
{{- toYaml $svc.resources | nindent 12 }}
securityContext:
{{- toYaml $svc.securityContext | nindent 12 }}
volumeMounts:
- name: data
mountPath: /data
{{- with $svc.volumeMounts }}
{{- toYaml . | nindent 12 }}
{{- end }}
startupProbe:
httpGet:
path: {{ $svc.startupProbe.httpGet.path }}
port: {{ $svc.startupProbe.httpGet.port }}
initialDelaySeconds: {{ $svc.startupProbe.initialDelaySeconds }}
periodSeconds: {{ $svc.startupProbe.periodSeconds }}
failureThreshold: {{ $svc.startupProbe.failureThreshold }}
livenessProbe:
httpGet:
path: {{ $svc.livenessProbe.httpGet.path }}
port: {{ $svc.livenessProbe.httpGet.port }}
periodSeconds: {{ $svc.livenessProbe.periodSeconds }}
readinessProbe:
httpGet:
path: {{ $svc.readinessProbe.httpGet.path }}
port: {{ $svc.readinessProbe.httpGet.port }}
periodSeconds: {{ $svc.readinessProbe.periodSeconds }}
volumes:
- name: data
persistentVolumeClaim:
claimName: app-data-pvc
{{- with $svc.volumes }}
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with $svc.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with $svc.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with $svc.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}

View File

@ -0,0 +1,38 @@
{{- range $svcName := list "gitdata" "gitpod" "gitsync" "email" }}
{{- $svcCfg := index $.Values $svcName }}
{{- $hpaCfg := index $.Values.autoscaling $svcName }}
{{- if and $svcCfg.enabled $hpaCfg.enabled }}
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: {{ include "app.serviceFullname" (dict "root" $ "name" $svcName) }}
namespace: {{ include "app.namespace" $ }}
labels:
{{- include "app.serviceLabels" (dict "root" $ "name" $svcName) | nindent 4 }}
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: {{ include "app.serviceFullname" (dict "root" $ "name" $svcName) }}
minReplicas: {{ $hpaCfg.minReplicas }}
maxReplicas: {{ $hpaCfg.maxReplicas }}
metrics:
{{- if $hpaCfg.targetCPUUtilizationPercentage }}
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: {{ $hpaCfg.targetCPUUtilizationPercentage }}
{{- end }}
{{- if $hpaCfg.targetMemoryUtilizationPercentage }}
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: {{ $hpaCfg.targetMemoryUtilizationPercentage }}
{{- end }}
{{- end }}
{{- end }}

View File

@ -0,0 +1,61 @@
{{- if .Values.ingress.enabled }}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ include "app.fullname" . }}
namespace: {{ include "app.namespace" . }}
labels:
{{- include "app.labels" . | nindent 4 }}
{{- with .Values.ingress.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
ingressClassName: {{ .Values.ingress.className }}
{{- if .Values.ingress.api.tls }}
tls:
{{- range .Values.ingress.api.tls }}
- hosts:
{{- range .hosts }}
- {{ . | quote }}
{{- end }}
secretName: {{ .secretName }}
{{- end }}
{{- range .Values.ingress.git.tls }}
- hosts:
{{- range .hosts }}
- {{ . | quote }}
{{- end }}
secretName: {{ .secretName }}
{{- end }}
{{- end }}
rules:
{{- range .Values.ingress.api.hosts }}
- host: {{ .host | quote }}
http:
paths:
{{- range .paths }}
- path: {{ .path }}
pathType: {{ .pathType }}
backend:
service:
name: {{ include "app.serviceFullname" (dict "root" $ "name" "gitdata") }}
port:
number: {{ $.Values.gitdata.service.port }}
{{- end }}
{{- end }}
{{- range .Values.ingress.git.hosts }}
- host: {{ .host | quote }}
http:
paths:
{{- range .paths }}
- path: {{ .path }}
pathType: {{ .pathType }}
backend:
service:
name: {{ include "app.serviceFullname" (dict "root" $ "name" "gitpod") }}-http
port:
number: {{ $.Values.gitpod.service.port }}
{{- end }}
{{- end }}
{{- end }}

View File

@ -0,0 +1,21 @@
{{- if .Values.podDisruptionBudget.enabled }}
{{- range $svcName := list "gitdata" "gitpod" "gitsync" "email" }}
{{- $svcCfg := index $.Values $svcName }}
{{- $pdbCfg := index $.Values.podDisruptionBudget $svcName }}
{{- if and $svcCfg.enabled $pdbCfg.minAvailable }}
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
name: {{ include "app.serviceFullname" (dict "root" $ "name" $svcName) }}
namespace: {{ $.Values.global.namespace | default $.Release.Namespace }}
labels:
{{- include "app.serviceLabels" (dict "root" $ "name" $svcName) | nindent 4 }}
spec:
minAvailable: {{ $pdbCfg.minAvailable }}
selector:
matchLabels:
{{- include "app.serviceSelectorLabels" (dict "root" $ "name" $svcName) | nindent 6 }}
{{- end }}
{{- end }}
{{- end }}

View File

@ -0,0 +1,141 @@
{{/*
Generate Services for each enabled service.
*/}}
{{- if .Values.gitdata.enabled }}
---
apiVersion: v1
kind: Service
metadata:
name: {{ include "app.serviceFullname" (dict "root" . "name" "gitdata") }}
namespace: {{ include "app.namespace" . }}
labels:
{{- include "app.serviceLabels" (dict "root" . "name" "gitdata") | nindent 4 }}
{{- with .Values.gitdata.service.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
type: {{ .Values.gitdata.service.type }}
ports:
- port: {{ .Values.gitdata.service.port }}
targetPort: http
protocol: TCP
name: http
selector:
{{- include "app.serviceSelectorLabels" (dict "root" . "name" "gitdata") | nindent 4 }}
{{- end }}
{{- if .Values.gitpod.enabled }}
---
apiVersion: v1
kind: Service
metadata:
name: {{ include "app.serviceFullname" (dict "root" . "name" "gitpod") }}-http
namespace: {{ include "app.namespace" . }}
labels:
{{- include "app.serviceLabels" (dict "root" . "name" "gitpod") | nindent 4 }}
{{- with .Values.gitpod.service.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
type: {{ .Values.gitpod.service.type }}
ports:
- port: {{ .Values.gitpod.service.port | default 8080 }}
targetPort: http
protocol: TCP
name: http
selector:
{{- include "app.serviceSelectorLabels" (dict "root" . "name" "gitpod") | nindent 4 }}
---
apiVersion: v1
kind: Service
metadata:
name: {{ include "app.serviceFullname" (dict "root" . "name" "gitpod") }}-ssh
namespace: {{ include "app.namespace" . }}
labels:
{{- include "app.serviceLabels" (dict "root" . "name" "gitpod") | nindent 4 }}
{{- with .Values.gitpod.sshService.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
type: {{ .Values.gitpod.sshService.type }}
ports:
- port: {{ .Values.gitpod.sshService.port | default 2222 }}
targetPort: ssh
protocol: TCP
name: ssh
selector:
{{- include "app.serviceSelectorLabels" (dict "root" . "name" "gitpod") | nindent 4 }}
---
apiVersion: v1
kind: Service
metadata:
name: {{ include "app.serviceFullname" (dict "root" . "name" "gitpod") }}-rpc
namespace: {{ include "app.namespace" . }}
labels:
{{- include "app.serviceLabels" (dict "root" . "name" "gitpod") | nindent 4 }}
{{- with .Values.gitpod.rpcService.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
type: {{ .Values.gitpod.rpcService.type }}
ports:
- port: {{ .Values.gitpod.rpcService.port | default 50051 }}
targetPort: grpc
protocol: TCP
name: grpc
selector:
{{- include "app.serviceSelectorLabels" (dict "root" . "name" "gitpod") | nindent 4 }}
{{- end }}
{{- if .Values.gitsync.enabled }}
---
apiVersion: v1
kind: Service
metadata:
name: {{ include "app.serviceFullname" (dict "root" . "name" "gitsync") }}
namespace: {{ include "app.namespace" . }}
labels:
{{- include "app.serviceLabels" (dict "root" . "name" "gitsync") | nindent 4 }}
{{- with .Values.gitsync.service.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
type: {{ .Values.gitsync.service.type }}
ports:
- port: {{ .Values.gitsync.service.port | default 8081 }}
targetPort: health
protocol: TCP
name: health
selector:
{{- include "app.serviceSelectorLabels" (dict "root" . "name" "gitsync") | nindent 4 }}
{{- end }}
{{- if and .Values.email.enabled .Values.email.service.enabled }}
---
apiVersion: v1
kind: Service
metadata:
name: {{ include "app.serviceFullname" (dict "root" . "name" "email") }}
namespace: {{ include "app.namespace" . }}
labels:
{{- include "app.serviceLabels" (dict "root" . "name" "email") | nindent 4 }}
{{- with .Values.email.service.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
type: {{ .Values.email.service.type }}
ports:
- port: {{ .Values.email.service.port | default 8083 }}
targetPort: health
protocol: TCP
name: health
selector:
{{- include "app.serviceSelectorLabels" (dict "root" . "name" "email") | nindent 4 }}
{{- end }}

View File

@ -0,0 +1,13 @@
{{- if .Values.serviceAccount.create }}
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ include "app.serviceAccountName" . }}
namespace: {{ include "app.namespace" . }}
labels:
{{- include "app.labels" . | nindent 4 }}
{{- with .Values.serviceAccount.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
{{- end }}

View File

@ -0,0 +1,36 @@
{{- if .Values.serviceMonitor.enabled }}
{{- $svcNames := list "gitdata" "gitpod" "gitsync" "email" }}
{{- range $svcName := $svcNames }}
{{- $svcCfg := index $.Values $svcName }}
{{- $monitorCfg := index $.Values.serviceMonitor.services $svcName }}
{{- if and $svcCfg.enabled $monitorCfg }}
---
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: {{ include "app.serviceFullname" (dict "root" $ "name" $svcName) }}
namespace: {{ include "app.namespace" $ }}
labels:
{{- include "app.serviceLabels" (dict "root" $ "name" $svcName) | nindent 4 }}
{{- with $.Values.serviceMonitor.labels }}
{{- toYaml . | nindent 4 }}
{{- end }}
{{- with $.Values.serviceMonitor.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
endpoints:
- interval: {{ $.Values.serviceMonitor.interval }}
port: {{ if eq $svcName "gitdata" }}http{{ else if eq $svcName "gitpod" }}http{{ else }}health{{ end }}
{{- if eq $svcName "gitdata" }}
path: /metrics
{{- else }}
path: /health
{{- end }}
selector:
matchLabels:
{{- include "app.serviceSelectorLabels" (dict "root" $ "name" $svcName) | nindent 6 }}
{{- end }}
{{- end }}
{{- end }}

365
chart/app/values.yaml Normal file
View File

@ -0,0 +1,365 @@
global:
image:
registry: "harbor.gitdata.me/app"
pullPolicy: IfNotPresent
tag: "latest"
imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
namespace: "gitdataai"
serviceAccount:
create: true
annotations: {}
name: ""
gitdata:
enabled: true
replicaCount: 1
image:
name: gitdata-gitdata
registry: ""
tag: ""
env:
APP_API_PORT: "8080"
APP_OTEL_SERVICE_NAME: "gitdata-api"
APP_GIT_RPC_ADDR: ""
APP_GIT_RPC_PORT: "50051"
service:
type: ClusterIP
port: 8080
annotations: {}
resources:
requests:
cpu: 250m
memory: 256Mi
limits:
cpu: 1000m
memory: 1Gi
startupProbe:
httpGet:
path: /metrics
port: http
initialDelaySeconds: 5
periodSeconds: 5
failureThreshold: 30
livenessProbe:
httpGet:
path: /metrics
port: http
periodSeconds: 15
readinessProbe:
httpGet:
path: /metrics
port: http
periodSeconds: 10
podAnnotations:
prometheus.io/scrape: "true"
prometheus.io/port: "8080"
prometheus.io/path: "/metrics"
podSecurityContext:
runAsNonRoot: true
runAsUser: 1000
fsGroup: 1000
securityContext:
readOnlyRootFilesystem: false
allowPrivilegeEscalation: false
nodeSelector: {}
tolerations: []
affinity: {}
volumes: []
volumeMounts: []
gitpod:
enabled: true
replicaCount: 1
image:
name: gitdata-gitpod
registry: ""
tag: ""
env:
APP_GIT_HTTP_PORT: "8080"
APP_SSH_PORT: "2222"
APP_GIT_RPC_ADDR: "0.0.0.0"
APP_GIT_RPC_PORT: "50051"
APP_OTEL_SERVICE_NAME: "gitpod"
APP_SSH_DOMAIN: ""
APP_GIT_HTTP_DOMAIN: ""
APP_REPOS_ROOT: "/data/repos"
service:
type: ClusterIP
port: 8080
annotations: {}
sshService:
type: LoadBalancer
port: 2222
annotations: {}
rpcService:
type: ClusterIP
port: 50051
annotations: {}
resources:
requests:
cpu: 500m
memory: 512Mi
limits:
cpu: 2000m
memory: 2Gi
startupProbe:
httpGet:
path: /health
port: http
initialDelaySeconds: 5
periodSeconds: 5
failureThreshold: 30
livenessProbe:
httpGet:
path: /health
port: http
periodSeconds: 20
readinessProbe:
httpGet:
path: /health
port: http
periodSeconds: 15
podAnnotations: {}
podSecurityContext:
runAsNonRoot: true
runAsUser: 1000
fsGroup: 1000
securityContext:
readOnlyRootFilesystem: false
allowPrivilegeEscalation: false
nodeSelector: {}
tolerations: []
affinity: {}
# -- SSH host key secret (mount to /etc/ssh)
sshHostKeySecret: ""
# -- Data volumes (repos storage)
volumes: []
volumeMounts: []
gitsync:
enabled: true
replicaCount: 1
image:
name: gitdata-gitsync
registry: ""
tag: ""
env:
APP_GITSYNC_HEALTH_PORT: "8081"
APP_OTEL_SERVICE_NAME: "gitsync"
APP_REPOS_ROOT: "/data/repos"
service:
type: ClusterIP
port: 8081
annotations: {}
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 500m
memory: 512Mi
startupProbe:
httpGet:
path: /health
port: health
initialDelaySeconds: 5
periodSeconds: 5
failureThreshold: 30
livenessProbe:
httpGet:
path: /health
port: health
periodSeconds: 30
readinessProbe:
httpGet:
path: /health
port: health
periodSeconds: 15
podAnnotations: {}
podSecurityContext:
runAsNonRoot: true
runAsUser: 1000
fsGroup: 1000
securityContext:
readOnlyRootFilesystem: false
allowPrivilegeEscalation: false
nodeSelector: {}
tolerations: []
affinity: {}
volumes: []
volumeMounts: []
email:
enabled: true
replicaCount: 1
image:
name: gitdata-email
registry: ""
tag: ""
env:
APP_EMAIL_HEALTH_PORT: "8083"
APP_OTEL_SERVICE_NAME: "email-service"
service:
enabled: false
type: ClusterIP
port: 8083
annotations: {}
resources:
requests:
cpu: 50m
memory: 64Mi
limits:
cpu: 200m
memory: 256Mi
startupProbe:
httpGet:
path: /health
port: health
initialDelaySeconds: 5
periodSeconds: 5
failureThreshold: 30
livenessProbe:
httpGet:
path: /health
port: health
periodSeconds: 30
readinessProbe:
httpGet:
path: /health
port: health
periodSeconds: 15
podAnnotations: {}
podSecurityContext:
runAsNonRoot: true
runAsUser: 1000
fsGroup: 1000
securityContext:
readOnlyRootFilesystem: false
allowPrivilegeEscalation: false
nodeSelector: {}
tolerations: []
affinity: {}
volumes: []
volumeMounts: []
ingress:
enabled: true
className: "nginx"
annotations:
cert-manager.io/cluster-issuer: "cloudflare-acme-cluster-issuer"
api:
hosts:
- host: dev.gitdata.ai
paths:
- path: /
pathType: Prefix
tls:
- hosts:
- dev.gitdata.ai
secretName: dev-gitdata-ai-tls
git:
hosts:
- host: gitdev.gitdata.ai
paths:
- path: /
pathType: Prefix
tls:
- hosts:
- gitdev.gitdata.ai
secretName: gitdev-gitdata-ai-tls
serviceMonitor:
enabled: false
interval: 30s
labels: {}
annotations: {}
services:
gitdata: true
gitpod: true
gitsync: true
email: true
autoscaling:
gitdata:
enabled: false
minReplicas: 1
maxReplicas: 10
targetCPUUtilizationPercentage: 80
targetMemoryUtilizationPercentage: ""
gitpod:
enabled: false
minReplicas: 1
maxReplicas: 5
targetCPUUtilizationPercentage: 75
targetMemoryUtilizationPercentage: ""
gitsync:
enabled: false
minReplicas: 1
maxReplicas: 5
targetCPUUtilizationPercentage: 80
targetMemoryUtilizationPercentage: ""
email:
enabled: false
minReplicas: 1
maxReplicas: 3
targetCPUUtilizationPercentage: 80
targetMemoryUtilizationPercentage: ""
podDisruptionBudget:
enabled: false
gitdata:
minAvailable: 1
gitpod:
minAvailable: 1
gitsync:
minAvailable: ""
email:
minAvailable: ""

View File

@ -1,6 +1,6 @@
{
"$schema": "https://ui.shadcn.com/schema.json",
"style": "radix-nova",
"style": "base-nova",
"rsc": false,
"tsx": true,
"tailwind": {
@ -12,6 +12,8 @@
},
"iconLibrary": "lucide",
"rtl": false,
"menuColor": "default",
"menuAccent": "subtle",
"aliases": {
"components": "@/components",
"utils": "@/lib/utils",
@ -19,9 +21,8 @@
"lib": "@/lib",
"hooks": "@/hooks"
},
"menuColor": "default",
"menuAccent": "subtle",
"registries": {
"@ai-elements": "https://ai-sdk.dev/elements/api/registry/{name}.json"
"@manifest": "https://ui.manifest.build/r/{name}.json",
"@kokonutui": "https://kokonutui.com/r/{name}.json"
}
}

View File

@ -1,90 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
# ── helpers ──────────────────────────────────────────────────────────
RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'; NC='\033[0m'
log() { echo -e "${GREEN}[OK]${NC} $*"; }
warn() { echo -e "${YELLOW}[WARN]${NC} $*"; }
err() { echo -e "${RED}[ERR]${NC} $*"; exit 1; }
command_exists() { command -v "$1" &>/dev/null; }
# ── defaults ─────────────────────────────────────────────────────────
NAMESPACE="${NAMESPACE:-app}"
RELEASE="${RELEASE:-deploy}"
CHART_DIR="${CHART_DIR:-./deploy}"
REGISTRY="${REGISTRY:-harbor.gitdata.me/gtateam}"
TAG="${TAG:-$(git rev-parse --short HEAD)}"
CONFIG_MAP="${CONFIG_MAP:-app-env}"
PVC_NAME="${PVC_NAME:-shared-data}"
# ── prerequisites ────────────────────────────────────────────────────
command_exists helm || err "helm not found — install via https://helm.sh/docs/intro/install/"
command_exists kubectl || err "kubectl not found — install via https://kubernetes.io/docs/tasks/tools/"
log "helm $(helm version --short)"
log "kubectl $(kubectl version --client --short 2>/dev/null || kubectl version -o json 2>/dev/null | grep gitVersion)"
# ── 1. Ensure namespace (not managed by Helm — preserved on uninstall) ──
log "Ensuring namespace $NAMESPACE exists..."
kubectl create namespace "$NAMESPACE" --dry-run=client -o yaml | kubectl apply -f -
# ── 2. Ensure prerequisites ─────────────────────────────────────────
if ! kubectl get namespace "$NAMESPACE" &>/dev/null; then
err "Namespace '$NAMESPACE' not found — create it first: kubectl create namespace $NAMESPACE"
fi
if ! kubectl get configmap "$CONFIG_MAP" -n "$NAMESPACE" &>/dev/null; then
err "ConfigMap '$CONFIG_MAP' not found in namespace '$NAMESPACE' — create it first"
fi
if ! kubectl get pvc "$PVC_NAME" -n "$NAMESPACE" &>/dev/null; then
err "PVC '$PVC_NAME' not found in namespace '$NAMESPACE' — create it first"
fi
# Protect ConfigMap and PVC from accidental Helm deletion
kubectl annotate configmap "$CONFIG_MAP" -n "$NAMESPACE" helm.sh/resource-policy=keep --overwrite
kubectl annotate pvc "$PVC_NAME" -n "$NAMESPACE" helm.sh/resource-policy=keep --overwrite
# cert-manager ClusterIssuer
if ! kubectl get clusterissuer cloudflare-acme-cluster-issuer &>/dev/null; then
warn "ClusterIssuer 'cloudflare-acme-cluster-issuer' not found — TLS certificate issuance will fail"
fi
log "Prerequisites verified"
# ── 3. Lint chart ────────────────────────────────────────────────────
log "Linting Helm chart..."
helm lint "$CHART_DIR" || err "Helm lint failed"
# ── 4. Deploy ────────────────────────────────────────────────────────
log "Deploying release $RELEASE with tag $TAG..."
if ! helm upgrade --install "$RELEASE" "$CHART_DIR" \
--namespace "$NAMESPACE" \
--set imageRegistry="$REGISTRY" \
--set imageTag="$TAG" \
--set configMapName="$CONFIG_MAP" \
--timeout 5m; then
echo ""
err "Deployment FAILED — release preserved for debugging.
Debug commands:
helm status $RELEASE -n $NAMESPACE
kubectl get pods -n $NAMESPACE
kubectl logs -n app <pod-name> --previous
helm rollback $RELEASE -n $NAMESPACE (rollback to previous release)
helm uninstall $RELEASE -n $NAMESPACE (remove failed release)"
fi
log "Release $RELEASE deployed successfully"
# ── 5. Verify ────────────────────────────────────────────────────────
log "Checking deployment status..."
kubectl get deployments -n "$NAMESPACE" -l app.kubernetes.io/instance="$RELEASE"
kubectl get pods -n "$NAMESPACE" -l app.kubernetes.io/instance="$RELEASE"
kubectl get services -n "$NAMESPACE" -l app.kubernetes.io/instance="$RELEASE"
kubectl get ingress -n "$NAMESPACE"
log "Deployment complete"

View File

@ -1,25 +0,0 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/
# Secrets
.server.yaml

View File

@ -1,6 +0,0 @@
apiVersion: v2
name: deploy
description: Helm chart for the project backend services
type: application
version: 0.1.0
appVersion: "0.2.9"

View File

@ -1,209 +0,0 @@
# Deploy Helm Chart
Monolithic Helm chart for all backend services.
## Services
| Service | Port(s) | Replicas | HPA | Purpose |
|----------------------|-------------------------|----------|----------|---------------------------------------------|
| `app` | 3000 (HTTP) | 2 | 210 | Main API server |
| `gitserver` | 8021 (HTTP), 2222 (SSH) | 1 | 15 | Git HTTP + SSH server |
| `email_worker` | 8084 (HTTP) | 1 | disabled | Email queue consumer (single instance only) |
| `git_hook` | 8083 (HTTP) | 1 | 15 | Git hook worker pool |
| `metrics_aggregator` | 9090 (HTTP) | 1 | 15 | Prometheus scrape + Loki push |
| `static_server` | 8081 (HTTP) | 1 | 15 | Static file server (avatars, blobs, media) |
## Prerequisites
The following resources must exist in the cluster **before** installing the Helm chart. They are not managed by Helm —
install, upgrade, and uninstall of the chart will not touch them.
### 1. Namespace
```bash
kubectl create namespace app
```
### 2. PVC (aliyun-nfs-app, 200Ti, ReadWriteMany)
```bash
kubectl apply -f - <<'EOF'
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: shared-data
namespace: app
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 200Ti
storageClassName: aliyun-nfs-app
EOF
```
> The chart references this PVC by hardcoded name `shared-data`. This name is immutable — it cannot be changed via Helm
> values.
### 3. ConfigMap
```bash
kubectl apply -f - <<'EOF'
apiVersion: v1
kind: ConfigMap
metadata:
name: app-env
namespace: app
data:
APP_REPOS_ROOT: "/data/repos"
APP_AVATAR_PATH: "/data/avatars"
STORAGE_PATH: "/data/files"
STATIC_ROOT: "/data"
APP_LOG_LEVEL: "info"
APP_COOKIE_SECURE: "false"
APP_DOMAIN_URL: "https://your-domain.com"
APP_DATABASE_URL: "postgres://user:pass@postgres:5432/app"
APP_REDIS_URL: "redis://redis:6379"
APP_AI_BASIC_URL: "https://api.openai.com/v1"
APP_AI_API_KEY: "sk-..."
APP_SMTP_PASSWORD: "..."
APP_SESSION_SECRET: "min-32-byte-random-string..."
APP_SSH_SERVER_PRIVATE_KEY: "<hex-encoded-private-key>"
EOF
```
| Variable | Default / Example | Required |
|------------------------------|-----------------------------|-----------|
| `APP_REPOS_ROOT` | `/data/repos` | Yes |
| `APP_AVATAR_PATH` | `/data/avatars` | Yes |
| `STORAGE_PATH` | `/data/files` | Yes |
| `STATIC_ROOT` | `/data` | Yes |
| `APP_LOG_LEVEL` | `info` | No |
| `APP_COOKIE_SECURE` | `false` | No |
| `APP_DOMAIN_URL` | `https://your-domain.com` | Yes |
| `APP_DATABASE_URL` | `postgres://...` | **Yes** |
| `APP_REDIS_URL` | `redis://...` | **Yes** |
| `APP_AI_BASIC_URL` | `https://api.openai.com/v1` | **Yes** |
| `APP_AI_API_KEY` | `sk-...` | **Yes** |
| `APP_SMTP_PASSWORD` | `...` | **Yes** |
| `APP_SESSION_SECRET` | min 32 bytes | **Yes** |
| `APP_SSH_SERVER_PRIVATE_KEY` | hex-encoded PEM | **Yes** |
| `APP_SSH_PORT` | `2222` | Yes (k8s) |
> **SSH host key**: `APP_SSH_SERVER_PRIVATE_KEY` must be the hex-encoded Ed25519 private key PEM bytes.
> ```bash
> ssh-keygen -t ed25519 -f /tmp/ssh_host_key -N ""
> hexdump -v -e '/1 "%02x"' < /tmp/ssh_host_key
> ```
>
> **Session secret**: generate 48 random bytes:
> ```bash
> openssl rand -base64 48
> ```
>
> Override the ConfigMap name with `--set configMapName=your-cm-name`.
### 4. Verify prerequisites
```bash
kubectl get namespace app
kubectl get pvc -n app shared-data
kubectl get configmap -n app app-env
```
## Quick Start
```bash
helm template deploy ./deploy --namespace app --set imageRegistry=ghcr.io/your-org
helm lint ./deploy
# Install
helm upgrade --install deploy ./deploy \
--namespace app \
--set imageRegistry=ghcr.io/your-org \
--set imageTag=v0.2.9
```
## Storage
All services share a single PVC (`shared-data`) via `subPath` mounts:
| SubPath | Mount | Used By |
|-----------|-----------------|--------------------------|
| `repos` | `/data/repos` | app, gitserver, git-hook |
| `avatars` | `/data/avatars` | app |
| `files` | `/data/files` | app |
| `static` | `/data` | static-server |
Pods run as UID/GID `1000` and set `fsGroup: 1000` so Git processes can create temporary object
directories under bare repositories. If an existing PVC was previously written by another UID,
fix ownership once from a maintenance pod:
```bash
chown -R 1000:1000 /data/repos
chmod -R u+rwX,g+rwX /data/repos
```
## Autoscaling
All services except `email_worker` have HPA enabled by default. The email worker is fixed at 1 replica and must not be
scaled.
To adjust HPA bounds per service:
```bash
--set services.app.autoscaling.maxReplicas=20
--set services.app.autoscaling.targetCPUUtilization=70
```
To disable HPA for a service:
```bash
--set services.git_hook.autoscaling.enabled=false
```
## Ingress
```bash
helm upgrade --install deploy ./deploy \
--namespace app \
--set ingress.enabled=true \
--set ingress.className=nginx \
--set ingress.hosts[0].host=your-domain.com
```
## Dependencies
All services require these to be reachable from the cluster:
- PostgreSQL (via `APP_DATABASE_URL`)
- Redis (via `APP_REDIS_URL`)
- Git binary (included in all Docker images)
- OpenAI-compatible API (via `APP_AI_BASIC_URL` + `APP_AI_API_KEY`)
- Qdrant vector DB (via `APP_QDRANT_URL`)
- SMTP server (via `APP_SMTP_*`)
- Embedding model (via `APP_EMBED_MODEL_*`)
Optional dependencies with graceful degradation:
| Dependency | Variable | Fallback |
|----------------|-------------------------------|------------------|
| NATS JetStream | `NATS_URL` + `NATS_TOKEN` | Redis queue |
| Loki | `LOKI_URL` | Logs discarded |
| OTEL Collector | `OTEL_EXPORTER_OTLP_ENDPOINT` | Tracing disabled |
## Production Example
```bash
helm upgrade --install deploy ./deploy \
--namespace app \
--set imageRegistry=ghcr.io/your-org \
--set imageTag=v0.2.9 \
--set services.app.replicas=3 \
--set services.app.autoscaling.maxReplicas=20 \
--set ingress.enabled=true \
--set ingress.className=nginx \
--set ingress.hosts[0].host=your-domain.com \
--set configMapName=app-env
```

View File

@ -1,19 +0,0 @@
Project backend services deployed to namespace: {{ .Release.Namespace }}
Services:
{{- range $svcKey, $svcVal := .Values.services }}
{{ $svcKey | replace "_" "-" }}: {{ if $svcVal.ports }}{{ range $portName, $portNum := $svcVal.ports }}{{ $portName }}={{ $portNum }} {{ end }}{{ else }}port={{ $svcVal.port }}{{ end }} {{ if $svcVal.autoscaling.enabled }}(HPA: {{ $svcVal.autoscaling.minReplicas }}-{{ $svcVal.autoscaling.maxReplicas }}){{ else }}(static: {{ $svcVal.replicaCount }}){{ end }}
{{- end }}
To access the app locally:
kubectl port-forward -n {{ .Release.Namespace }} svc/{{ include "deploy.serviceFullname" (dict "root" . "svcKey" "app") }} 3000:3000
To check HPA status:
{{- range $svcKey, $svcVal := .Values.services }}
{{- if $svcVal.autoscaling.enabled }}
kubectl get hpa -n {{ $.Release.Namespace }} {{ include "deploy.serviceFullname" (dict "root" $ "svcKey" $svcKey) }}
{{- end }}
{{- end }}
To check all pods:
kubectl get pods -n {{ .Release.Namespace }} -l "app.kubernetes.io/name={{ include "deploy.name" . }}"

View File

@ -1,78 +0,0 @@
{{/*
Expand the name of the chart.
*/}}
{{- define "deploy.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Create a default fully qualified app name.
*/}}
{{- define "deploy.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}
{{/*
Service fullname — includes service key for per-service resources.
Underscores in svcKey are replaced with hyphens for valid Kubernetes names.
*/}}
{{- define "deploy.serviceFullname" -}}
{{- printf "%s-%s" (include "deploy.fullname" .root) (.svcKey | replace "_" "-") | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Chart name and version as used by the chart label.
*/}}
{{- define "deploy.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Common labels
*/}}
{{- define "deploy.labels" -}}
helm.sh/chart: {{ include "deploy.chart" . }}
{{ include "deploy.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}
{{/*
Selector labels
*/}}
{{- define "deploy.selectorLabels" -}}
app.kubernetes.io/name: {{ include "deploy.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}
{{/*
Per-service selector labels — used by Service to target the right Deployment.
Underscores in svcKey are replaced with hyphens for valid Kubernetes label values.
*/}}
{{- define "deploy.serviceSelectorLabels" -}}
app.kubernetes.io/name: {{ include "deploy.name" .root }}
app.kubernetes.io/instance: {{ .root.Release.Name }}
app.kubernetes.io/component: {{ .svcKey | replace "_" "-" }}
{{- end }}
{{/*
Create the name of the service account to use
*/}}
{{- define "deploy.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (include "deploy.fullname" .) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
{{- end }}

View File

@ -1,89 +0,0 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "deploy.serviceFullname" (dict "root" . "svcKey" "app") }}
labels:
{{- include "deploy.labels" . | nindent 4 }}
app.kubernetes.io/component: app
spec:
replicas: {{ .Values.services.app.replicaCount | default 1 }}
selector:
matchLabels:
{{- include "deploy.serviceSelectorLabels" (dict "root" . "svcKey" "app") | nindent 6 }}
template:
metadata:
labels:
{{- include "deploy.labels" . | nindent 8 }}
app.kubernetes.io/component: app
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
serviceAccountName: {{ include "deploy.serviceAccountName" . }}
{{- with .Values.podSecurityContext }}
securityContext:
{{- toYaml . | nindent 8 }}
{{- end }}
containers:
- name: app
{{- with .Values.securityContext }}
securityContext:
{{- toYaml . | nindent 12 }}
{{- end }}
image: "{{ .Values.imageRegistry }}/{{ .Values.services.app.repository }}:{{ .Values.imageTag | default .Chart.AppVersion }}"
imagePullPolicy: IfNotPresent
{{- with .Values.services.app.command }}
command:
{{- toYaml . | nindent 12 }}
{{- end }}
ports:
- name: http
containerPort: {{ .Values.services.app.port }}
protocol: TCP
envFrom:
- configMapRef:
name: {{ .Values.configMapName }}
{{- with .Values.services.app.extraEnv }}
env:
{{- range $key, $val := . }}
- name: {{ $key }}
value: {{ $val | quote }}
{{- end }}
{{- end }}
livenessProbe:
httpGet:
path: /health
port: http
initialDelaySeconds: 10
periodSeconds: 15
readinessProbe:
httpGet:
path: /health
port: http
initialDelaySeconds: 5
periodSeconds: 10
{{- with .Values.services.app.resources }}
resources:
{{- toYaml . | nindent 12 }}
{{- end }}
{{- with .Values.services.app.volumeMounts }}
volumeMounts:
{{- toYaml . | nindent 12 }}
{{- end }}
volumes:
- name: shared-data
persistentVolumeClaim:
claimName: shared-data
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}

View File

@ -1,16 +0,0 @@
apiVersion: v1
kind: Service
metadata:
name: {{ include "deploy.serviceFullname" (dict "root" . "svcKey" "app") }}
labels:
{{- include "deploy.labels" . | nindent 4 }}
app.kubernetes.io/component: app
spec:
type: ClusterIP
ports:
- port: {{ .Values.services.app.port }}
targetPort: http
protocol: TCP
name: http
selector:
{{- include "deploy.serviceSelectorLabels" (dict "root" . "svcKey" "app") | nindent 4 }}

View File

@ -1,70 +0,0 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "deploy.serviceFullname" (dict "root" . "svcKey" "email_worker") }}
labels:
{{- include "deploy.labels" . | nindent 4 }}
app.kubernetes.io/component: email-worker
spec:
replicas: {{ .Values.services.email_worker.replicaCount | default 1 }}
selector:
matchLabels:
{{- include "deploy.serviceSelectorLabels" (dict "root" . "svcKey" "email_worker") | nindent 6 }}
template:
metadata:
labels:
{{- include "deploy.labels" . | nindent 8 }}
app.kubernetes.io/component: email-worker
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
serviceAccountName: {{ include "deploy.serviceAccountName" . }}
{{- with .Values.podSecurityContext }}
securityContext:
{{- toYaml . | nindent 8 }}
{{- end }}
containers:
- name: email-worker
{{- with .Values.securityContext }}
securityContext:
{{- toYaml . | nindent 12 }}
{{- end }}
image: "{{ .Values.imageRegistry }}/{{ .Values.services.email_worker.repository }}:{{ .Values.imageTag | default .Chart.AppVersion }}"
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: {{ .Values.services.email_worker.port }}
protocol: TCP
envFrom:
- configMapRef:
name: {{ .Values.configMapName }}
livenessProbe:
httpGet:
path: /health
port: http
initialDelaySeconds: 10
periodSeconds: 15
readinessProbe:
httpGet:
path: /health
port: http
initialDelaySeconds: 5
periodSeconds: 10
{{- with .Values.services.email_worker.resources }}
resources:
{{- toYaml . | nindent 12 }}
{{- end }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}

View File

@ -1,16 +0,0 @@
apiVersion: v1
kind: Service
metadata:
name: {{ include "deploy.serviceFullname" (dict "root" . "svcKey" "email_worker") }}
labels:
{{- include "deploy.labels" . | nindent 4 }}
app.kubernetes.io/component: email-worker
spec:
type: ClusterIP
ports:
- port: {{ .Values.services.email_worker.port }}
targetPort: http
protocol: TCP
name: http
selector:
{{- include "deploy.serviceSelectorLabels" (dict "root" . "svcKey" "email_worker") | nindent 4 }}

View File

@ -1,78 +0,0 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "deploy.serviceFullname" (dict "root" . "svcKey" "git_hook") }}
labels:
{{- include "deploy.labels" . | nindent 4 }}
app.kubernetes.io/component: git-hook
spec:
replicas: {{ .Values.services.git_hook.replicaCount | default 1 }}
selector:
matchLabels:
{{- include "deploy.serviceSelectorLabels" (dict "root" . "svcKey" "git_hook") | nindent 6 }}
template:
metadata:
labels:
{{- include "deploy.labels" . | nindent 8 }}
app.kubernetes.io/component: git-hook
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
serviceAccountName: {{ include "deploy.serviceAccountName" . }}
{{- with .Values.podSecurityContext }}
securityContext:
{{- toYaml . | nindent 8 }}
{{- end }}
containers:
- name: git-hook
{{- with .Values.securityContext }}
securityContext:
{{- toYaml . | nindent 12 }}
{{- end }}
image: "{{ .Values.imageRegistry }}/{{ .Values.services.git_hook.repository }}:{{ .Values.imageTag | default .Chart.AppVersion }}"
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: {{ .Values.services.git_hook.port }}
protocol: TCP
envFrom:
- configMapRef:
name: {{ .Values.configMapName }}
livenessProbe:
httpGet:
path: /health
port: http
initialDelaySeconds: 10
periodSeconds: 15
readinessProbe:
httpGet:
path: /health
port: http
initialDelaySeconds: 5
periodSeconds: 10
{{- with .Values.services.git_hook.resources }}
resources:
{{- toYaml . | nindent 12 }}
{{- end }}
{{- with .Values.services.git_hook.volumeMounts }}
volumeMounts:
{{- toYaml . | nindent 12 }}
{{- end }}
volumes:
- name: shared-data
persistentVolumeClaim:
claimName: shared-data
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}

View File

@ -1,16 +0,0 @@
apiVersion: v1
kind: Service
metadata:
name: {{ include "deploy.serviceFullname" (dict "root" . "svcKey" "git_hook") }}
labels:
{{- include "deploy.labels" . | nindent 4 }}
app.kubernetes.io/component: git-hook
spec:
type: ClusterIP
ports:
- port: {{ .Values.services.git_hook.port }}
targetPort: http
protocol: TCP
name: http
selector:
{{- include "deploy.serviceSelectorLabels" (dict "root" . "svcKey" "git_hook") | nindent 4 }}

View File

@ -1,88 +0,0 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "deploy.serviceFullname" (dict "root" . "svcKey" "gitserver") }}
labels:
{{- include "deploy.labels" . | nindent 4 }}
app.kubernetes.io/component: gitserver
spec:
replicas: {{ .Values.services.gitserver.replicaCount | default 1 }}
selector:
matchLabels:
{{- include "deploy.serviceSelectorLabels" (dict "root" . "svcKey" "gitserver") | nindent 6 }}
template:
metadata:
labels:
{{- include "deploy.labels" . | nindent 8 }}
app.kubernetes.io/component: gitserver
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
serviceAccountName: {{ include "deploy.serviceAccountName" . }}
{{- with .Values.podSecurityContext }}
securityContext:
{{- toYaml . | nindent 8 }}
{{- end }}
containers:
- name: gitserver
{{- with .Values.securityContext }}
securityContext:
{{- toYaml . | nindent 12 }}
{{- end }}
image: "{{ .Values.imageRegistry }}/{{ .Values.services.gitserver.repository }}:{{ .Values.imageTag | default .Chart.AppVersion }}"
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: {{ .Values.services.gitserver.ports.http }}
protocol: TCP
- name: ssh
containerPort: {{ .Values.services.gitserver.ports.ssh }}
protocol: TCP
envFrom:
- configMapRef:
name: {{ .Values.configMapName }}
{{- with .Values.services.gitserver.extraEnv }}
env:
{{- range $key, $val := . }}
- name: {{ $key }}
value: {{ $val | quote }}
{{- end }}
{{- end }}
livenessProbe:
httpGet:
path: /health
port: http
initialDelaySeconds: 10
periodSeconds: 15
readinessProbe:
httpGet:
path: /health
port: http
initialDelaySeconds: 5
periodSeconds: 10
{{- with .Values.services.gitserver.resources }}
resources:
{{- toYaml . | nindent 12 }}
{{- end }}
{{- with .Values.services.gitserver.volumeMounts }}
volumeMounts:
{{- toYaml . | nindent 12 }}
{{- end }}
volumes:
- name: shared-data
persistentVolumeClaim:
claimName: shared-data
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}

View File

@ -1,20 +0,0 @@
apiVersion: v1
kind: Service
metadata:
name: {{ include "deploy.serviceFullname" (dict "root" . "svcKey" "gitserver") }}
labels:
{{- include "deploy.labels" . | nindent 4 }}
app.kubernetes.io/component: gitserver
spec:
type: ClusterIP
ports:
- port: {{ .Values.services.gitserver.ports.http }}
targetPort: http
protocol: TCP
name: http
- port: {{ .Values.services.gitserver.ports.ssh }}
targetPort: ssh
protocol: TCP
name: ssh
selector:
{{- include "deploy.serviceSelectorLabels" (dict "root" . "svcKey" "gitserver") | nindent 4 }}

View File

@ -1,21 +0,0 @@
apiVersion: v1
kind: Service
metadata:
name: {{ include "deploy.serviceFullname" (dict "root" . "svcKey" "gitserver") }}-ssh
labels:
{{- include "deploy.labels" . | nindent 4 }}
app.kubernetes.io/component: gitserver
annotations:
{{- with .Values.services.gitserver.sshService.annotations }}
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
type: LoadBalancer
externalTrafficPolicy: Local
ports:
- port: 22
targetPort: ssh
protocol: TCP
name: ssh
selector:
{{- include "deploy.serviceSelectorLabels" (dict "root" . "svcKey" "gitserver") | nindent 4 }}

View File

@ -1,26 +0,0 @@
{{- range $svcKey, $svcVal := .Values.services }}
{{- if $svcVal.autoscaling.enabled }}
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: {{ include "deploy.serviceFullname" (dict "root" $ "svcKey" $svcKey) }}
labels:
{{- include "deploy.labels" $ | nindent 4 }}
app.kubernetes.io/component: {{ $svcKey | replace "_" "-" }}
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: {{ include "deploy.serviceFullname" (dict "root" $ "svcKey" $svcKey) }}
minReplicas: {{ $svcVal.autoscaling.minReplicas }}
maxReplicas: {{ $svcVal.autoscaling.maxReplicas }}
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: {{ $svcVal.autoscaling.targetCPUUtilization }}
{{- end }}
{{- end }}

View File

@ -1,41 +0,0 @@
{{- if .Values.ingress.enabled -}}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ include "deploy.fullname" . }}
labels:
{{- include "deploy.labels" . | nindent 4 }}
{{- with .Values.ingress.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
{{- with .Values.ingress.className }}
ingressClassName: {{ . }}
{{- end }}
{{- if .Values.ingress.tls }}
tls:
{{- range .Values.ingress.tls }}
- hosts:
{{- range .hosts }}
- {{ . | quote }}
{{- end }}
secretName: {{ .secretName }}
{{- end }}
{{- end }}
rules:
{{- range .Values.ingress.hosts }}
- host: {{ .host | quote }}
http:
paths:
{{- range .paths }}
- path: {{ .path }}
pathType: {{ .pathType }}
backend:
service:
name: {{ include "deploy.serviceFullname" (dict "root" $ "svcKey" .serviceName) }}
port:
number: {{ .servicePort }}
{{- end }}
{{- end }}
{{- end }}

View File

@ -1,70 +0,0 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "deploy.serviceFullname" (dict "root" . "svcKey" "metrics_aggregator") }}
labels:
{{- include "deploy.labels" . | nindent 4 }}
app.kubernetes.io/component: metrics-aggregator
spec:
replicas: {{ .Values.services.metrics_aggregator.replicaCount | default 1 }}
selector:
matchLabels:
{{- include "deploy.serviceSelectorLabels" (dict "root" . "svcKey" "metrics_aggregator") | nindent 6 }}
template:
metadata:
labels:
{{- include "deploy.labels" . | nindent 8 }}
app.kubernetes.io/component: metrics-aggregator
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
serviceAccountName: {{ include "deploy.serviceAccountName" . }}
{{- with .Values.podSecurityContext }}
securityContext:
{{- toYaml . | nindent 8 }}
{{- end }}
containers:
- name: metrics-aggregator
{{- with .Values.securityContext }}
securityContext:
{{- toYaml . | nindent 12 }}
{{- end }}
image: "{{ .Values.imageRegistry }}/{{ .Values.services.metrics_aggregator.repository }}:{{ .Values.imageTag | default .Chart.AppVersion }}"
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: {{ .Values.services.metrics_aggregator.port }}
protocol: TCP
envFrom:
- configMapRef:
name: {{ .Values.configMapName }}
livenessProbe:
httpGet:
path: /health
port: http
initialDelaySeconds: 10
periodSeconds: 15
readinessProbe:
httpGet:
path: /health
port: http
initialDelaySeconds: 5
periodSeconds: 10
{{- with .Values.services.metrics_aggregator.resources }}
resources:
{{- toYaml . | nindent 12 }}
{{- end }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}

View File

@ -1,16 +0,0 @@
apiVersion: v1
kind: Service
metadata:
name: {{ include "deploy.serviceFullname" (dict "root" . "svcKey" "metrics_aggregator") }}
labels:
{{- include "deploy.labels" . | nindent 4 }}
app.kubernetes.io/component: metrics-aggregator
spec:
type: ClusterIP
ports:
- port: {{ .Values.services.metrics_aggregator.port }}
targetPort: http
protocol: TCP
name: http
selector:
{{- include "deploy.serviceSelectorLabels" (dict "root" . "svcKey" "metrics_aggregator") | nindent 4 }}

View File

@ -1 +0,0 @@
{{/* Secret disabled — all config via ConfigMap */}}

View File

@ -1,13 +0,0 @@
{{- if .Values.serviceAccount.create -}}
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ include "deploy.serviceAccountName" . }}
labels:
{{- include "deploy.labels" . | nindent 4 }}
{{- with .Values.serviceAccount.annotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
automountServiceAccountToken: {{ .Values.serviceAccount.automount }}
{{- end }}

Some files were not shown because too many files have changed in this diff Show More