docs: consolidate 53 files into 15 structured files in 3 folders
- business/ (5): club-commission, payment, ecommerce, membership, content - technical/ (5): cms-arch, ui, deployment, migration, api - overview/ (5): flowcharts, index, changelog, glossary, roadmap - Removed all old folders: backoffice, cms, deployment, docs, frontoffice, migration, ui-modernization, business (old) - Updated internal links with relative folder paths
This commit is contained in:
masoodafar-web
@@ -1,678 +0,0 @@
|
||||
# 🔧 راهنمای CI/CD Pipeline — Gitea Actions + K3s
|
||||
|
||||
> آخرین بروزرسانی: February 17, 2026
|
||||
|
||||
---
|
||||
|
||||
## 📐 معماری کلی
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ K3s Cluster (194.5.195.53) │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────┐ │
|
||||
│ │ gitea-runner Pod (2 containers) │ │
|
||||
│ │ │ │
|
||||
│ │ ┌──────────────────┐ ┌──────────────────────┐ │ │
|
||||
│ │ │ docker (DinD) │ │ runner (act_runner) │ │ │
|
||||
│ │ │ docker:dind │ │ gitea/act_runner │ │ │
|
||||
│ │ │ privileged: true │ │ DOCKER_HOST= │ │ │
|
||||
│ │ │ port: 2375 │ │ tcp://localhost:2375│ │ │
|
||||
│ │ └──────────────────┘ └──────────────────────┘ │ │
|
||||
│ │ ▲ shared volumes: docker-storage │ │
|
||||
│ └─────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌─────────────┐ ┌───────────┐ ┌────────────────┐ │
|
||||
│ │ Gitea │ │ Nexus │ │ Docker Reg. │ │
|
||||
│ │ :3000 │ │ :32081 │ │ :32082 (pull) │ │
|
||||
│ │ │ │ (NuGet) │ │ :30080 (push) │ │
|
||||
│ └─────────────┘ └───────────┘ └────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### سه لایه Docker-in-Docker:
|
||||
```
|
||||
K3s containerd (لایه ۱)
|
||||
└── gitea-runner Pod → docker container (DinD daemon) (لایه ۲)
|
||||
└── workflow: docker run / docker build (لایه ۳)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📁 فایلهای Workflow
|
||||
|
||||
### 🐳 K8s Pipelines (Docker + K3s) — آفلاین
|
||||
|
||||
| سرویس | فایل | Branch | Image | Deploy |
|
||||
|--------|------|--------|-------|--------|
|
||||
| CMS | `kub-deploy.yml` | `kub-stage` | `admin/cms` | SSH → kubectl |
|
||||
| BackOffice | `kub-deploy.yml` | `kub-stage` | `admin/backoffice` | SSH → kubectl |
|
||||
| FrontOffice | `kub-deploy.yml` | `kub-stage` | `admin/frontoffice` | SSH → kubectl |
|
||||
| CMS | `prod-deploy.yml` | `production` | `admin/cms:prod` | SSH → kubectl |
|
||||
| BackOffice | `prod-deploy.yml` | `production` | `admin/backoffice:prod` | SSH → kubectl |
|
||||
| FrontOffice | `prod-deploy.yml` | `production` | `admin/frontoffice:prod` | SSH → kubectl |
|
||||
|
||||
### 🪟 Windows/IIS Pipelines (Legacy) — آنلاین
|
||||
|
||||
| سرویس | فایل | Branch | Target |
|
||||
|--------|------|--------|--------|
|
||||
| CMS | `cms-stage.yml` | `stage_new` | IIS → `cms.kbs1.ir` |
|
||||
| BackOffice | `bo-stage.yml` | `stage-new` | IIS → `admin.kbs1.ir` |
|
||||
| FrontOffice | `fo-stage.yml` | `stage-new` | IIS → `kbs1.ir` |
|
||||
|
||||
> ⚠️ Stage pipeline ها از Windows runner + IIS استفاده میکنن و Docker ندارن.
|
||||
|
||||
### ساختار مشترک Pipeline:
|
||||
```
|
||||
1. Start Docker daemon (DinD)
|
||||
2. Checkout code (git clone)
|
||||
3. Login to Docker registries (32082 + 30080)
|
||||
4. [CMS only] Publish Protobuf packages
|
||||
5. Build Docker Image
|
||||
6. Push to Registry
|
||||
7. Deploy to Kubernetes (SSH → kubectl rollout restart)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🐛 مشکلات حلشده و راهحلها
|
||||
|
||||
### مشکل ۱: `iptables failed: Permission denied`
|
||||
|
||||
**خطا:**
|
||||
```
|
||||
iptables v1.8.10 (nf_tables): Could not fetch rule set generation id: Permission denied
|
||||
```
|
||||
|
||||
**علت:** K3s containerd به Docker daemon اجازه تغییر iptables نمیده.
|
||||
|
||||
**راهحل:** غیرفعال کردن networking در dockerd:
|
||||
```bash
|
||||
dockerd --iptables=false --ip6tables=false --bridge=none --storage-driver=vfs &
|
||||
```
|
||||
|
||||
> ⚠️ با `--bridge=none` نیاز به شبکهسازی Docker نیست چون فقط build و push انجام میشه.
|
||||
|
||||
---
|
||||
|
||||
### مشکل ۲: `failed to unmount overlayfs: operation not permitted`
|
||||
|
||||
**خطا:**
|
||||
```
|
||||
failed to register layer: unshare: operation not permitted
|
||||
```
|
||||
|
||||
**علت:** `overlay2` storage driver نیاز به mount namespace داره که داخل K3s مجاز نیست.
|
||||
|
||||
**راهحل:** استفاده از `vfs` storage driver:
|
||||
```bash
|
||||
dockerd --storage-driver=vfs &
|
||||
```
|
||||
|
||||
> ⚠️ `vfs` کندتره ولی هیچ mount syscall خاصی نیاز نداره. برای CI/CD کافیه.
|
||||
|
||||
---
|
||||
|
||||
### مشکل ۳: `no basic auth credentials` هنگام pull ایمیج
|
||||
|
||||
**خطا:**
|
||||
```
|
||||
Error response from daemon: Head "https://194.5.195.53:32082/v2/dotnet/sdk/manifests/9.0":
|
||||
no basic auth credentials
|
||||
```
|
||||
|
||||
**علت:** `docker login` فقط قبل از push انجام میشد، ولی `docker build` (یا `docker run`) هم از `32082` ایمیج pull میکنه.
|
||||
|
||||
**راهحل:** اضافه کردن step "Login to Docker registries" بلافاصله بعد از Checkout:
|
||||
```yaml
|
||||
- name: Login to Docker registries
|
||||
run: |
|
||||
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login 194.5.195.53:32082 -u admin --password-stdin
|
||||
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login ${{ env.REGISTRY }} -u admin --password-stdin
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### مشکل ۴: `unshare: operation not permitted` هنگام extract لایهها
|
||||
|
||||
**خطا:**
|
||||
```
|
||||
docker: failed to register layer: unshare: operation not permitted
|
||||
```
|
||||
|
||||
**علت اصلی (دو بخش):**
|
||||
|
||||
**بخش ۱:** Gitea act_runner دیفالت `container.privileged: false` داره. یعنی job container ها بدون privileged ساخته میشن — حتی اگه workflow بنویسه `options: --privileged`.
|
||||
|
||||
**بخش ۲:** env var `CONFIG_FILE` در runner container ست نبود → `run.sh` فلگ `--config` رو به `act_runner daemon` پاس نمیداد → config.yaml اصلاً لود نمیشد!
|
||||
|
||||
**راهحل (سمت سرور):**
|
||||
|
||||
۱. ساخت ConfigMap:
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: runner-config
|
||||
namespace: default
|
||||
data:
|
||||
config.yaml: |
|
||||
log:
|
||||
level: info
|
||||
runner:
|
||||
file: .runner
|
||||
capacity: 1
|
||||
timeout: 3h
|
||||
container:
|
||||
privileged: true
|
||||
options: "--security-opt seccomp=unconfined --security-opt apparmor=unconfined"
|
||||
valid_volumes:
|
||||
- "**"
|
||||
```
|
||||
|
||||
۲. Mount کردن در Deployment + env var:
|
||||
```bash
|
||||
kubectl patch deployment gitea-runner --type=json -p='[
|
||||
{"op":"add","path":"/spec/template/spec/containers/1/env/-",
|
||||
"value":{"name":"CONFIG_FILE","value":"/data/config.yaml"}},
|
||||
{"op":"add","path":"/spec/template/spec/containers/1/volumeMounts/-",
|
||||
"value":{"name":"runner-config","mountPath":"/data/config.yaml","subPath":"config.yaml"}},
|
||||
{"op":"add","path":"/spec/template/spec/volumes/-",
|
||||
"value":{"name":"runner-config","configMap":{"name":"runner-config"}}}
|
||||
]'
|
||||
```
|
||||
|
||||
> ⚠️ **نکته مهم:** بدون `CONFIG_FILE=/data/config.yaml` env var، فایل `run.sh` داخل act_runner image فلگ `--config` رو پاس نمیده!
|
||||
|
||||
---
|
||||
|
||||
### مشکل ۵: Protobuf restore از nuget.org بجای Nexus
|
||||
|
||||
**علت:** `dotnet restore` بدون `--configfile` از دیفالت NuGet sources استفاده میکنه.
|
||||
|
||||
**راهحل:**
|
||||
```bash
|
||||
dotnet restore "$proj" --configfile src/NuGet.config
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### مشکل ۶: عدم دسترسی شبکه با `--bridge=none`
|
||||
|
||||
**علت:** `dockerd --bridge=none` شبکه Docker bridge رو غیرفعال میکنه. در نتیجه container هایی که با `docker run` یا `docker build` ساخته میشن، دسترسی شبکه ندارن (مثلاً `dotnet restore` نمیتونه به Nexus وصل بشه).
|
||||
|
||||
**راهحل:** استفاده از `--network host` در `docker run` و `docker build`:
|
||||
```bash
|
||||
# Protobuf step
|
||||
docker run --rm --network host -v $(pwd):/src -w /src ...
|
||||
|
||||
# Build step
|
||||
DOCKER_BUILDKIT=0 docker build --network host -t ... .
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### مشکل ۷: `failed to prepare ... as ...: invalid argument` (BuildKit)
|
||||
|
||||
**خطا:**
|
||||
```
|
||||
ERROR: failed to build: failed to solve: failed to prepare xxx as yyy: invalid argument
|
||||
```
|
||||
|
||||
**علت:** BuildKit (بیلدر پیشفرض Docker ≥23) از snapshotter overlay استفاده میکنه که با `--storage-driver=vfs` سازگاری نداره.
|
||||
|
||||
**راهحل:** غیرفعال کردن BuildKit:
|
||||
```bash
|
||||
DOCKER_BUILDKIT=0 docker build --network host -t ... .
|
||||
```
|
||||
|
||||
> ⚠️ Legacy builder از vfs بدون مشکل استفاده میکنه.
|
||||
|
||||
---
|
||||
|
||||
### مشکل ۸: `COPY libs/` fails in Docker build (BackOffice)
|
||||
|
||||
**خطا:**
|
||||
```
|
||||
COPY failed: file not found in build context: stat libs/: file does not exist
|
||||
```
|
||||
|
||||
**علت:** Dockerfile خط `COPY ["libs/", "libs/"]` داشت ولی `libs/` خارج از Docker build context (`src/`) بود. قبلاً BFF DLLها استفاده میشدن، ولی حالا از NuGet package مستقیم استفاده میشه.
|
||||
|
||||
**راهحل:**
|
||||
1. حذف `COPY ["libs/", "libs/"]` از Dockerfile
|
||||
2. تغییر `ProjectReference` به `PackageReference` در csproj:
|
||||
```xml
|
||||
<!-- قبل -->
|
||||
<ProjectReference Include="../../../CMS/src/CMSMicroservice.Protobuf/CMSMicroservice.Protobuf.csproj" />
|
||||
<!-- بعد -->
|
||||
<PackageReference Include="Foursat.CMSMicroservice.Protobuf" Version="0.0.178" />
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### مشکل ۹: ProjectReference خارج از Docker context (FrontOffice/BackOffice)
|
||||
|
||||
**خطا:**
|
||||
```
|
||||
error CS0246: The type or namespace name 'CustomerAddressModel' could not be found
|
||||
```
|
||||
|
||||
**علت:** csproj از `ProjectReference Include="../../../CMS/src/CMSMicroservice.Protobuf/..."` استفاده میکرد. در Docker build context فقط `src/` موجوده → CMS قابل دسترسی نیست.
|
||||
|
||||
**راهحل:**
|
||||
1. بامپ نسخه پروتوباف (`0.0.177` → `0.0.178`)
|
||||
2. `dotnet pack -c Release` و push به Nexus
|
||||
3. تغییر هر دو پروژه (FrontOffice + BackOffice) به `PackageReference`
|
||||
|
||||
```bash
|
||||
# Pack & Push
|
||||
cd CMS/src/CMSMicroservice.Protobuf
|
||||
dotnet pack -c Release
|
||||
dotnet nuget push bin/Release/Foursat.CMSMicroservice.Protobuf.0.0.178.nupkg \
|
||||
--source http://194.5.195.53:32081/repository/foursat-nuget-hosted/index.json \
|
||||
--api-key admin:87zH26nbqT --skip-duplicate
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### مشکل ۱۰: `nginx:alpine` TLS handshake timeout
|
||||
|
||||
**خطا:**
|
||||
```
|
||||
Get "https://registry-1.docker.io/v2/": net/http: TLS handshake timeout
|
||||
```
|
||||
|
||||
**علت:** Dockerfile خط `FROM nginx:alpine` مستقیم از Docker Hub پول میکرد ولی سرور به Docker Hub دسترسی نداره.
|
||||
|
||||
**راهحل:** تغییر به رجیستری لوکال:
|
||||
```dockerfile
|
||||
# قبل
|
||||
FROM nginx:alpine AS final
|
||||
# بعد
|
||||
FROM 194.5.195.53:32082/nginx:alpine AS final
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### مشکل ۱۱: SERVER_PASSWORD secret missing → Permission denied
|
||||
|
||||
**خطا:**
|
||||
```
|
||||
Permission denied, please try again.
|
||||
```
|
||||
|
||||
**علت:** سکرت `SERVER_PASSWORD` در ریپو Gitea تنظیم نشده بود. Pipeline از `sshpass -e` با `${{ secrets.SERVER_PASSWORD }}` برای SSH استفاده میکنه.
|
||||
|
||||
**راهحل:** اضافه کردن سکرت از طریق Gitea API:
|
||||
```bash
|
||||
curl -sk -u "admin:87zH26nbqT" -X PUT \
|
||||
"https://git.se.kbs1.ir/api/v1/repos/admin/BackOffice/actions/secrets/SERVER_PASSWORD" \
|
||||
-H "Content-Type: application/json" -d '{"data":"87zH26nbqT"}'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### مشکل ۱۲: CMS ingress 502 — backend-protocol: GRPC
|
||||
|
||||
**خطا:** `https://cms.se.kbs1.ir/` → 502 Bad Gateway
|
||||
|
||||
**علت:** CMS ingress annotation `backend-protocol: GRPC` داشت + Kestrel فقط `Http2`. مرورگر HTTP/1.1 میفرسته → nginx نمیتونه به gRPC backend فوروارد کنه.
|
||||
|
||||
**راهحل (دو تغییر):**
|
||||
1. Kestrel protocol → `Http1AndHttp2` (هم gRPC هم REST):
|
||||
```bash
|
||||
kubectl set env deployment/cms Kestrel__EndpointDefaults__Protocols=Http1AndHttp2
|
||||
```
|
||||
2. حذف GRPC annotations از ingress:
|
||||
```bash
|
||||
kubectl annotate ingress cms-ingress nginx.ingress.kubernetes.io/backend-protocol-
|
||||
kubectl annotate ingress cms-ingress nginx.ingress.kubernetes.io/grpc-backend-
|
||||
```
|
||||
|
||||
> ⚠️ FrontOffice از gRPC-Web استفاده میکنه که روی HTTP/1.1 هم کار میکنه.
|
||||
|
||||
---
|
||||
|
||||
## 🔄 تغییرات prod-deploy (قدیم → جدید)
|
||||
|
||||
| مورد | قدیم (prod-deploy) | جدید |
|
||||
|------|-------------------|------|
|
||||
| Container image | `docker:latest` | `docker-sshpass:latest` (شامل sshpass + git) |
|
||||
| Proxy | `HTTP_PROXY` + `HTTPS_PROXY` | حذف شد (آفلاین) |
|
||||
| Registry | `gitea-svc:3000` + external | فقط `194.5.195.53:30080` |
|
||||
| kubectl | `apk add` + `curl` از اینترنت | SSH → `kubectl` مستقیم روی سرور |
|
||||
| Auth | hardcoded password | `secrets.REGISTRY_PASSWORD` + `secrets.SERVER_PASSWORD` |
|
||||
| BuildKit | فعال (دیفالت) | `DOCKER_BUILDKIT=0` |
|
||||
| Network | Docker bridge (دیفالت) | `--network host` |
|
||||
| dockerd | دیفالت | `--iptables=false --ip6tables=false --bridge=none --storage-driver=vfs` |
|
||||
| Deploy | `KUBECONFIG_PROD` (base64) | SSH + sshpass (مثل kub-stage) |
|
||||
|
||||
> ✅ حالا همه ۶ K8s pipeline (۳ stage + ۳ prod) از **یک الگوی مشترک آفلاین** استفاده میکنن.
|
||||
|
||||
---
|
||||
|
||||
containers:
|
||||
- name: docker # DinD sidecar
|
||||
image: 194.5.195.53:32082/docker:dind
|
||||
securityContext:
|
||||
privileged: true
|
||||
env:
|
||||
- DOCKER_TLS_CERTDIR: ""
|
||||
volumeMounts:
|
||||
- /var/lib/docker → docker-storage
|
||||
- /etc/docker/daemon.json → docker-config (ConfigMap)
|
||||
|
||||
- name: runner # Gitea act_runner
|
||||
image: 194.5.195.53:32082/gitea/act_runner:latest
|
||||
env:
|
||||
- GITEA_INSTANCE_URL: http://gitea-svc:3000
|
||||
- DOCKER_HOST: tcp://localhost:2375
|
||||
- CONFIG_FILE: /data/config.yaml # ← حیاتی! بدون این runner config لود نمیشه
|
||||
volumeMounts:
|
||||
- /data → runner-data
|
||||
- /data/config.yaml → runner-config (ConfigMap)
|
||||
```
|
||||
|
||||
### ConfigMaps:
|
||||
| نام | محتوا | Mount Path |
|
||||
|-----|-------|------------|
|
||||
| `docker-daemon-config` | `daemon.json` با insecure-registries | `/etc/docker/daemon.json` |
|
||||
| `runner-config` | `config.yaml` با privileged + seccomp | `/data/config.yaml` |
|
||||
|
||||
### Labels (ثبتشده در Gitea):
|
||||
```
|
||||
ubuntu-latest → docker://docker.gitea.com/runner-images:ubuntu-latest
|
||||
ubuntu-24.04 → docker://docker.gitea.com/runner-images:ubuntu-24.04
|
||||
ubuntu-22.04 → docker://docker.gitea.com/runner-images:ubuntu-22.04
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔑 Secrets مورد نیاز (Gitea → Settings → Secrets)
|
||||
|
||||
| Secret | استفاده |
|
||||
|--------|---------|
|
||||
| `REGISTRY_PASSWORD` | پسورد Docker registry (admin) |
|
||||
| `SERVER_PASSWORD` | پسورد SSH سرور (root) — ⚠️ باید در هر ۳ ریپو ست بشه |
|
||||
|
||||
> **نکته:** اگر `SERVER_PASSWORD` ست نباشه، مرحله Deploy با `Permission denied` فیل میشه.
|
||||
> با API اضافه کنید:
|
||||
> ```bash
|
||||
> curl -sk -u "admin:PASSWORD" -X PUT \
|
||||
> "https://git.se.kbs1.ir/api/v1/repos/admin/REPO/actions/secrets/SERVER_PASSWORD" \
|
||||
> -H "Content-Type: application/json" -d '{"data":"PASSWORD"}'
|
||||
> ```
|
||||
|
||||
---
|
||||
|
||||
## 🔧 dockerd فلگهای نهایی
|
||||
|
||||
```bash
|
||||
dockerd --iptables=false --ip6tables=false --bridge=none --storage-driver=vfs &
|
||||
```
|
||||
|
||||
| Flag | دلیل |
|
||||
|------|-------|
|
||||
| `--iptables=false` | K3s اجازه تغییر iptables نمیده |
|
||||
| `--ip6tables=false` | مشابه بالا برای IPv6 |
|
||||
| `--bridge=none` | نیازی به Docker bridge network نیست |
|
||||
| `--storage-driver=vfs` | overlay2 نمیتونه mount کنه داخل K3s |
|
||||
|
||||
---
|
||||
|
||||
## 🔍 عیبیابی Pipeline
|
||||
|
||||
### ۱. چک وضعیت Runner:
|
||||
```bash
|
||||
# SSH به سرور
|
||||
ssh root@194.5.195.53
|
||||
|
||||
# آیا runner pod بالاست؟
|
||||
kubectl get pods -l app=gitea-runner
|
||||
|
||||
# لاگ runner
|
||||
kubectl logs <pod-name> -c runner --tail=30
|
||||
|
||||
# لاگ DinD
|
||||
kubectl logs <pod-name> -c docker --tail=30
|
||||
```
|
||||
|
||||
### ۲. تست Docker داخل Runner:
|
||||
```bash
|
||||
# exec به DinD container
|
||||
kubectl exec <pod-name> -c docker -- docker info
|
||||
|
||||
# آیا registry قابل دسترسیه؟
|
||||
kubectl exec <pod-name> -c docker -- docker pull 194.5.195.53:32082/dotnet/sdk:9.0
|
||||
```
|
||||
|
||||
### ۳. چک config runner:
|
||||
```bash
|
||||
# آیا config.yaml mount شده؟
|
||||
kubectl exec <pod-name> -c runner -- cat /data/config.yaml
|
||||
|
||||
# آیا privileged فعاله؟
|
||||
kubectl exec <pod-name> -c docker -- docker inspect <job-container> \
|
||||
--format '{{.HostConfig.Privileged}} {{.HostConfig.SecurityOpt}}'
|
||||
```
|
||||
|
||||
### ۴. ریاستارت runner:
|
||||
```bash
|
||||
kubectl rollout restart deployment/gitea-runner
|
||||
kubectl rollout status deployment/gitea-runner --timeout=120s
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📋 Workflow Template (کامل)
|
||||
|
||||
```yaml
|
||||
name: Build and Deploy to Kubernetes
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- kub-stage
|
||||
|
||||
env:
|
||||
REGISTRY: 194.5.195.53:30080
|
||||
IMAGE_NAME: admin/<service-name>
|
||||
K8S_SERVER: 194.5.195.53
|
||||
|
||||
jobs:
|
||||
build-and-deploy:
|
||||
runs-on: ubuntu-latest
|
||||
container:
|
||||
image: 194.5.195.53:32082/docker-sshpass:latest
|
||||
options: --privileged
|
||||
steps:
|
||||
- name: Start Docker daemon
|
||||
run: |
|
||||
mkdir -p /etc/docker
|
||||
cat > /etc/docker/daemon.json << 'DAEMON'
|
||||
{
|
||||
"insecure-registries": ["194.5.195.53:30080", "194.5.195.53:32500", "194.5.195.53:32082"]
|
||||
}
|
||||
DAEMON
|
||||
dockerd --iptables=false --ip6tables=false --bridge=none --storage-driver=vfs &
|
||||
for i in $(seq 1 90); do
|
||||
if docker info >/dev/null 2>&1; then
|
||||
echo "✅ Docker ready"; break
|
||||
fi
|
||||
sleep 2
|
||||
done
|
||||
|
||||
- name: Checkout code
|
||||
run: |
|
||||
git clone --depth 1 --branch kub-stage http://gitea-svc:3000/admin/<repo>.git .
|
||||
|
||||
- name: Login to Docker registries
|
||||
run: |
|
||||
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login 194.5.195.53:32082 -u admin --password-stdin
|
||||
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login ${{ env.REGISTRY }} -u admin --password-stdin
|
||||
|
||||
- name: Build Docker Image
|
||||
run: |
|
||||
DOCKER_BUILDKIT=0 docker build --network host -t ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest .
|
||||
|
||||
- name: Push to Registry
|
||||
run: |
|
||||
docker push ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest
|
||||
|
||||
- name: Deploy to Kubernetes
|
||||
run: |
|
||||
export SSHPASS="${{ secrets.SERVER_PASSWORD }}"
|
||||
sshpass -e ssh -o StrictHostKeyChecking=no root@${{ env.K8S_SERVER }} "
|
||||
kubectl rollout restart deployment/<service>
|
||||
kubectl rollout status deployment/<service> --timeout=180s
|
||||
"
|
||||
```
|
||||
---
|
||||
|
||||
## 🐛 باگ بحرانی: Cross-Deployment — Push به Production ریدیپلوی Staging (اسفند ۱۴۰۴)
|
||||
|
||||
### علائم:
|
||||
- Push به برنچ `production` → هم production و هم staging ریدیپلوی شدند
|
||||
- CMS staging pod بعد از push ریستارت شد
|
||||
- Runner log: **۲ تسک CMS** بجای ۱ تسک اجرا شد
|
||||
|
||||
### علت ریشهای:
|
||||
Gitea Act Runner **تمام فایلهای workflow** داخل `.gitea/workflows/` برنچ push شده رو اجرا میکنه — حتی اگه `on.push.branches` برنچ دیگهای باشه. وقتی production push شد، `kub-deploy.yml` (trigger: `kub-stage`) هم اجرا شد و ایمیج `admin/cms:latest` رو با کد production ساخت → staging از `latest` pull کرد → **staging با DB production بالا اومد!**
|
||||
|
||||
### راهحل:
|
||||
حذف workflowهای staging از برنچ production (هر ۳ ریپو):
|
||||
```bash
|
||||
git rm .gitea/workflows/kub-deploy.yml .gitea/workflows/cms-stage.yml # CMS
|
||||
git rm .gitea/workflows/fo-stage.yml .gitea/workflows/kub-deploy.yml # FrontOffice
|
||||
git rm .gitea/workflows/bo-stage.yml .gitea/workflows/kub-deploy.yml # BackOffice
|
||||
```
|
||||
|
||||
> ⚠️ **قانون طلایی:** هر برنچ فقط workflow مربوط به خودش رو داشته باشه.
|
||||
|
||||
---
|
||||
|
||||
## 🐛 مشکل ۱۳: Production deploy ایمیج pull نمیشد
|
||||
|
||||
**علت:** Production K8s از `git.foursat.afrino.co/admin/cms:prod` pull میکرد، ولی CI ایمیج رو به `194.5.195.53:30080` push میکرد.
|
||||
|
||||
**راهحل:**
|
||||
1. اضافه کردن `194.5.195.53:30080` به `/etc/rancher/k3s/registries.yaml` پروداکشن + ریاستارت K3s
|
||||
2. آپدیت deployment image: `kubectl set image deployment/cms cms=194.5.195.53:30080/admin/cms:prod`
|
||||
3. فیکس `prod-deploy.yml`: `K8S_SSH_PASSWORD` → `SERVER_PASSWORD`, `rollout restart` → `set image :sha`
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Production Workflow Template (فعلی)
|
||||
|
||||
```yaml
|
||||
name: Build and Deploy to Production
|
||||
on:
|
||||
push:
|
||||
branches: [production]
|
||||
env:
|
||||
REGISTRY: 194.5.195.53:30080
|
||||
IMAGE_NAME: admin/<service>
|
||||
K8S_SERVER: 45.149.79.127
|
||||
jobs:
|
||||
build-and-deploy:
|
||||
runs-on: ubuntu-latest
|
||||
container:
|
||||
image: 194.5.195.53:32082/docker-sshpass:latest
|
||||
options: --privileged
|
||||
steps:
|
||||
# ... (Start Docker, Checkout, Login — مشابه staging)
|
||||
- name: Build Docker Image
|
||||
run: |
|
||||
DOCKER_BUILDKIT=0 docker build --network host \
|
||||
-t ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }} \
|
||||
-t ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:prod .
|
||||
- name: Push to Registry
|
||||
run: |
|
||||
docker push ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
|
||||
docker push ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:prod
|
||||
- name: Deploy to Production
|
||||
run: |
|
||||
export SSHPASS="${{ secrets.SERVER_PASSWORD }}"
|
||||
sshpass -e ssh -o StrictHostKeyChecking=no root@${{ env.K8S_SERVER }} "
|
||||
kubectl set image deployment/<svc> <svc>=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
|
||||
kubectl rollout status deployment/<svc> --timeout=300s
|
||||
"
|
||||
```
|
||||
|
||||
> تفاوت staging vs production: staging = tag `latest` + `rollout restart` | production = tag `sha` + `set image`
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ مشخصات دو محیط
|
||||
|
||||
| | Staging | Production |
|
||||
|--|---------|------------|
|
||||
| **سرور** | `194.5.195.53` | `45.149.79.127` |
|
||||
| **DB** | `mssql-svc@Foursat` | `45.149.79.127,31433@KBS` |
|
||||
| **Registry** | `194.5.195.53:30080` (local) | همان staging registry |
|
||||
| **Image Tags** | `:latest` | `:prod` + `:sha` |
|
||||
| **Branch** | `kub-stage` | `production` |
|
||||
| **Domains** | `*.se.kbs1.ir` | `*.kbs1.ir` |
|
||||
|
||||
---
|
||||
|
||||
## 🐛 مشکل ۱۴: K8S_SERVER اشتباه در prod-deploy.yml (CMS + BackOffice)
|
||||
|
||||
**تاریخ:** February 17, 2026
|
||||
**علت:** `K8S_SERVER` در `prod-deploy.yml` CMS و BackOffice هنوز `194.5.195.53` (staging) بود بجای `45.149.79.127` (production).
|
||||
**عارضه:** `kubectl set image` به سرور staging ارسال میشد — deployment production آپدیت نمیشد.
|
||||
|
||||
**فایلهای فیکس شده:**
|
||||
- `CMS/.gitea/workflows/prod-deploy.yml` — `K8S_SERVER: 194.5.195.53` → `45.149.79.127`
|
||||
- `BackOffice/.gitea/workflows/prod-deploy.yml` — `K8S_SERVER: 194.5.195.53` → `45.149.79.127`
|
||||
- FrontOffice قبلاً درست بود ✅
|
||||
|
||||
**فیکس اضافی:** هر دو workflow از `kubectl rollout restart` به `kubectl set image` تغییر کردن تا ایمیج SHA-tagged واقعاً set بشه.
|
||||
|
||||
---
|
||||
|
||||
## 🐛 مشکل ۱۵: نبود appsettings.Production.json — URLهای staging روی production
|
||||
|
||||
**تاریخ:** February 17, 2026
|
||||
**علت:** هیچکدوم از ۳ پروژه `appsettings.Production.json` نداشتن. از طرفی `ASPNETCORE_ENVIRONMENT=Production` ست بود → fallback به `appsettings.json` (که URLهای staging داشت).
|
||||
|
||||
**عارضهها:**
|
||||
- CMS: `CmsBaseUrl=cms.se.kbs1.ir` → ZarinPal callback به staging برمیگشت
|
||||
- CMS: `FrontOfficeBaseUrl=foursat.se.kbs1.ir` → redirect بعد از پرداخت به staging میرفت
|
||||
- FrontOffice: `GwUrl=localhost:32846` → gRPC به هیچجا وصل نمیشد
|
||||
- BackOffice: `GwUrl=localhost:32847` → gRPC به هیچجا وصل نمیشد
|
||||
|
||||
**فایلهای ساخته شده:**
|
||||
|
||||
| پروژه | فایل | محتوای کلیدی |
|
||||
|--------|------|-------------|
|
||||
| CMS | `src/CMSMicroservice.WebApi/appsettings.Production.json` | `CmsBaseUrl=cms.kbs1.ir`, `FrontOfficeBaseUrl=kbs1.ir`, `DB=KBS`, `ZarinPal.UseSandbox=false` |
|
||||
| FrontOffice | `src/FrontOffice.Main/appsettings.Production.json` | `GwUrl=cms.kbs1.ir` |
|
||||
| BackOffice | `src/BackOffice/wwwroot/appsettings.Production.json` | `GwUrl=cms.kbs1.ir` |
|
||||
|
||||
> ⚠️ **نکته:** CMS فایل `appsettings.Production.json` در `.gitignore` هست (`**/ appsettings.Production.json`). با `git add -f` ترک شد. بعد از هر تغییر باید دوباره force add بشه.
|
||||
|
||||
---
|
||||
|
||||
## 🐛 مشکل ۱۶: nginx image path اشتباه در BackOffice Dockerfile (production branch)
|
||||
|
||||
**تاریخ:** February 17, 2026
|
||||
**علت:** Dockerfile روی برنچ `production` از `194.5.195.53:32082/library/nginx:alpine` استفاده میکرد که در رجیستری وجود نداشت. روی `kub-stage` قبلاً فیکس شده بود ولی merge به production این خط رو override کرده بود.
|
||||
|
||||
**ارور CI:**
|
||||
```
|
||||
Step 9/14 : FROM 194.5.195.53:32082/library/nginx:alpine AS final
|
||||
manifest for 194.5.195.53:32082/library/nginx:alpine not found: manifest unknown
|
||||
```
|
||||
|
||||
**رفع:**
|
||||
```dockerfile
|
||||
# قبل (اشتباه)
|
||||
FROM 194.5.195.53:32082/library/nginx:alpine AS final
|
||||
|
||||
# بعد (صحیح)
|
||||
FROM 194.5.195.53:32082/nginx:alpine AS final
|
||||
```
|
||||
|
||||
**نکته:** این فیکس مستقیماً روی برنچ `production` انجام و push شد (کامیت `743403e`).
|
||||
@@ -1,161 +0,0 @@
|
||||
# 🚀 FourSat Offline Deployment
|
||||
|
||||
این دایرکتوری شامل scripts و مستندات برای deploy کردن سرویسهای FourSat **بدون نیاز به اینترنت** است.
|
||||
|
||||
## ⚡ Quick Start
|
||||
|
||||
### گزینه 1: Setup خودکار (پیشنهادی)
|
||||
|
||||
**روی ماشینی با Docker و اینترنت:**
|
||||
|
||||
```bash
|
||||
cd /home/masoud/Apps/project/FourSat/deployment
|
||||
./setup-offline-complete.sh
|
||||
```
|
||||
|
||||
این اسکریپت:
|
||||
- ✅ Images را pull میکند
|
||||
- ✅ Export میکند به `.tar`
|
||||
- ✅ به server transfer میکند
|
||||
- ✅ در containerd server import میکند
|
||||
- ✅ تمام!
|
||||
|
||||
### گزینه 2: Setup دستی
|
||||
|
||||
اگر Docker روی این ماشین نیست، مستندات کامل را بخوانید:
|
||||
|
||||
```bash
|
||||
cat OFFLINE-SETUP-GUIDE.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📁 فایلهای مهم
|
||||
|
||||
### 🎯 اسکریپتهای اصلی
|
||||
- **`setup-offline-complete.sh`** ⭐ - Setup کامل خودکار (نیاز به Docker)
|
||||
- **`export-import-images.sh`** - Export/Import دستی images
|
||||
- **`OFFLINE-SETUP-GUIDE.md`** 📚 - راهنمای کامل قدم به قدم
|
||||
|
||||
### 📖 مستندات
|
||||
- **`README.md`** - این فایل
|
||||
- **`QUICK-REFERENCE.md`** - مرجع سریع commands
|
||||
- **`DEPLOYMENT-GUIDE.md`** - راهنمای deployment
|
||||
- **`SERVER-SETUP-GUIDE.md`** - تنظیمات server
|
||||
- **`CHANGES-SUMMARY.md`** - لیست تغییرات انجام شده
|
||||
- **`SETUP-INSTRUCTIONS.md`** - دستورالعملهای setup
|
||||
|
||||
### 🛠️ اسکریپتهای کمکی
|
||||
- `pull-base-images.sh` - Pull کردن base images
|
||||
- `save-images.sh` - ذخیره images به tar
|
||||
- `load-images.sh` - بارگذاری images از tar
|
||||
- `cache-nuget-packages.sh` - Cache کردن NuGet packages
|
||||
- `build-all-offline.sh` - Build تمام سرویسها offline
|
||||
- `cleanup.sh` - پاکسازی فایلهای build
|
||||
- `test-services.sh` - تست سرویسهای deploy شده
|
||||
- `k8s-deploy.sh` - Deploy به Kubernetes
|
||||
- `k8s-health-check.sh` - بررسی سلامت deployments
|
||||
- `quick-start.sh` - Setup تعاملی
|
||||
- `server-cache-images.sh` - Cache images روی server
|
||||
- `load-cached-images.sh` - بارگذاری cached images
|
||||
- `verify-cache.sh` - بررسی cache
|
||||
- `pull-images-to-node.sh` - Pull به Kubernetes node
|
||||
|
||||
---
|
||||
|
||||
## 🔍 چکلیست Setup
|
||||
|
||||
- [ ] **Base images در containerd server** (روی node نه pod)
|
||||
- [ ] **Workflows بدون proxy** تنظیم شدهاند
|
||||
- [ ] **DNS = "0.0.0.0"** برای جلوگیری از دسترسی به اینترنت
|
||||
- [ ] **DOCKER_BUILDKIT=0** برای استفاده از legacy builder
|
||||
- [ ] **Test یک workflow** برای اطمینان از کار کردن offline
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ معماری Deployment
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ Kubernetes Cluster (194.5.195.53) │
|
||||
│ │
|
||||
│ ┌───────────────────────────────────┐ │
|
||||
│ │ Node (containerd) │ │
|
||||
│ │ ├─ mcr.microsoft.com/dotnet/* │ │
|
||||
│ │ └─ nginx:alpine │ │
|
||||
│ └───────────────────────────────────┘ │
|
||||
│ ▲ │
|
||||
│ │ images available to pods │
|
||||
│ │ │
|
||||
│ ┌──────┴──────────────────────────┐ │
|
||||
│ │ gitea-runner pod │ │
|
||||
│ │ ├─ dind (Docker-in-Docker) │ │
|
||||
│ │ └─ runner (Gitea Actions) │ │
|
||||
│ └─────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**کلید موفقیت**: Images باید در **containerd روی node** باشند، نه در pod!
|
||||
|
||||
---
|
||||
|
||||
## 🐛 عیبیابی
|
||||
|
||||
### ❌ Workflow هنوز download میکند؟
|
||||
|
||||
```bash
|
||||
# بررسی images روی server
|
||||
ssh root@194.5.195.53 "ctr -n k8s.io images list | grep mcr.microsoft.com"
|
||||
```
|
||||
|
||||
اگر خالی است، setup را دوباره اجرا کنید.
|
||||
|
||||
### ❌ Build با error میخورد؟
|
||||
|
||||
```bash
|
||||
# بررسی logs workflow در Gitea
|
||||
# اگر "TLS handshake timeout" میبینید → DNS هنوز active است
|
||||
# اگر "image not found" میبینید → images در containerd نیستند
|
||||
```
|
||||
|
||||
### ❌ Import به containerd کار نمیکند?
|
||||
|
||||
```bash
|
||||
# مطمئن شوید namespace درست است
|
||||
ssh root@194.5.195.53 "ctr namespaces list"
|
||||
|
||||
# باید k8s.io را ببینید
|
||||
# برای import همیشه از -n k8s.io استفاده کنید:
|
||||
ctr -n k8s.io images import /path/to/image.tar
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📞 Support
|
||||
|
||||
مشکل دارید? مستندات کامل را بخوانید:
|
||||
|
||||
```bash
|
||||
# راهنمای کامل setup offline
|
||||
cat OFFLINE-SETUP-GUIDE.md
|
||||
|
||||
# مرجع سریع commands
|
||||
cat QUICK-REFERENCE.md
|
||||
|
||||
# راهنمای deployment
|
||||
cat DEPLOYMENT-GUIDE.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ✨ ویژگیها
|
||||
|
||||
- ✅ **کاملاً Offline** - هیچ download در حین build
|
||||
- ✅ **خودکار** - یک اسکریپت برای setup کامل
|
||||
- ✅ **مستند شده** - راهنماهای جامع فارسی
|
||||
- ✅ **تست شده** - روی Kubernetes با k3s
|
||||
- ✅ **سریع** - build بدون تاخیر network
|
||||
|
||||
---
|
||||
|
||||
**نوشته شده برای پروژه FourSat** 🛰️
|
||||
@@ -1,570 +0,0 @@
|
||||
# FourSat Infrastructure Deployment Guide
|
||||
|
||||
## 📌 Server Information
|
||||
|
||||
### سرور Staging
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Server IP | `194.5.195.53` |
|
||||
| SSH Access | `root / 87zH26nbqT` |
|
||||
| Kubernetes | K3s with local-path storage |
|
||||
| ServiceLB | K3s svclb (built-in) |
|
||||
| Domains | `*.se.kbs1.ir` |
|
||||
|
||||
### سرور Production
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Server IP | `45.149.79.127` |
|
||||
| SSH Access | `root / 87zH26nbqT` |
|
||||
| Kubernetes | K3s with local-path storage |
|
||||
| ServiceLB | K3s svclb (built-in) |
|
||||
| Domains | `*.kbs1.ir` |
|
||||
|
||||
---
|
||||
|
||||
## 🗄️ Database (MSSQL Server 2022)
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Image | `mssql/server:2022-CU16` |
|
||||
| Nexus Image | `194.5.195.53:32082/mcr.microsoft.com/mssql/server:2022-CU16` |
|
||||
| SA Password | `87zH26nbqT` |
|
||||
| Service | `mssql-svc:1433` |
|
||||
| PVC | `mssql-pvc` (10Gi) |
|
||||
|
||||
### Databases:
|
||||
|
||||
#### Staging (194.5.195.53):
|
||||
- `gitea` - Gitea metadata
|
||||
- `Foursat` - Application database (staging)
|
||||
- `Hosein` - Application database
|
||||
|
||||
#### Production (45.149.79.127):
|
||||
- `KBS` - Application database (production)
|
||||
|
||||
### Connection Strings:
|
||||
```
|
||||
# Staging
|
||||
Server=mssql-svc,1433;Database=Foursat;User Id=sa;Password=87zH26nbqT;TrustServerCertificate=true
|
||||
|
||||
# Production (appsettings.Production.json)
|
||||
Server=mssql-svc;Database=KBS;User Id=sa;Password=YourStrong@Passw0rd;TrustServerCertificate=True
|
||||
|
||||
# Production (env override — قدیمی، از بیرون cluster)
|
||||
# Server=45.149.79.127,31433;Database=KBS;User Id=sa;Password=YourStrong@Passw0rd;TrustServerCertificate=true
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📦 Git Server (Gitea)
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Image | `gitea/gitea:1.25.3` |
|
||||
| Nexus Image | `194.5.195.53:32082/gitea/gitea:1.25.3` |
|
||||
| Admin User | `admin` |
|
||||
| Admin Email | `admin@afrino.co` |
|
||||
| Service | `gitea-svc:3000` |
|
||||
| PVC | `gitea-pvc` (10Gi) |
|
||||
| Database | MSSQL (`gitea` database) |
|
||||
|
||||
### Repositories:
|
||||
- `admin/cms.git`
|
||||
- `admin/backoffice.git`
|
||||
- `admin/backoffice.bff.git`
|
||||
- `admin/frontoffice.git`
|
||||
- `admin/frontoffice.bff.git`
|
||||
- `admin/docs.git`
|
||||
|
||||
---
|
||||
|
||||
## 📚 Package Registry (Nexus)
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Image | `sonatype/nexus3:3.38.0` |
|
||||
| UI Port | `32081` (NodePort) |
|
||||
| Docker Registry Port | `32082` (NodePort, HTTP) |
|
||||
| PVC | `nexus-data-pvc` (50Gi) |
|
||||
|
||||
### Usage:
|
||||
```bash
|
||||
# Tag and push image
|
||||
ctr -n k8s.io images tag <source> 194.5.195.53:32082/<name>:<tag>
|
||||
ctr -n k8s.io images push --plain-http 194.5.195.53:32082/<name>:<tag>
|
||||
|
||||
# List images
|
||||
curl http://194.5.195.53:32082/v2/_catalog
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🌐 Ingress (ingress-nginx)
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| Image | `registry.k8s.io/ingress-nginx/controller:v1.14.1` |
|
||||
| Nexus Image | `194.5.195.53:32082/registry.k8s.io/ingress-nginx/controller:v1.14.1` |
|
||||
| HTTP Port | `80` |
|
||||
| HTTPS Port | `443` |
|
||||
|
||||
### ⚠️ CRITICAL WARNING:
|
||||
**DO NOT use `hostNetwork: true` with K3s svclb!**
|
||||
|
||||
K3s uses svclb (ServiceLB) for LoadBalancer services. If you add `hostNetwork: true`:
|
||||
- Both svclb pods AND ingress-nginx pods will try to bind to ports 80/443
|
||||
- This causes conflicts and connection failures
|
||||
- svclb is already exposing ports correctly
|
||||
|
||||
See: `deployment/docs/INGRESS-NGINX-WARNING.md`
|
||||
|
||||
---
|
||||
|
||||
## 💾 Persistent Volume Claims
|
||||
|
||||
| PVC Name | Size | Status | Reclaim Policy |
|
||||
|----------|------|--------|----------------|
|
||||
| `mssql-pvc` | 10Gi | Bound | Retain |
|
||||
| `gitea-pvc` | 10Gi | Bound | Retain |
|
||||
| `nexus-data-pvc` | 50Gi | Bound | Retain |
|
||||
| `seq-pvc` | 5Gi | Bound | Retain |
|
||||
|
||||
### Storage Location (K3s local-path):
|
||||
```
|
||||
/var/lib/rancher/k3s/storage/pvc-<uuid>_default_<pvc-name>/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔄 Backup Strategy
|
||||
|
||||
### Automatic Backup (CronJob):
|
||||
- Runs daily at 2:00 AM
|
||||
- Backs up: gitea, Foursat, Hosein databases
|
||||
- Retention: 7 days
|
||||
- Location: `/backups/` on mssql-pvc
|
||||
|
||||
### Manual Backup:
|
||||
```bash
|
||||
# Trigger manual backup
|
||||
kubectl create job --from=cronjob/mssql-backup mssql-backup-manual-$(date +%s)
|
||||
|
||||
# Or apply the manual job
|
||||
kubectl apply -f k8s-manifests/mssql-backup-cronjob.yaml
|
||||
```
|
||||
|
||||
### Restore Database:
|
||||
```bash
|
||||
# Exec into MSSQL pod
|
||||
kubectl exec -it deploy/mssql -- /bin/bash
|
||||
|
||||
# Restore
|
||||
/opt/mssql-tools18/bin/sqlcmd -S localhost -U sa -P '87zH26nbqT' -C -Q "RESTORE DATABASE [Foursat] FROM DISK = '/backups/Foursat_YYYYMMDD_HHMMSS.bak' WITH REPLACE"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Deployment Commands
|
||||
|
||||
### Deploy All:
|
||||
```bash
|
||||
# Apply manifests
|
||||
kubectl apply -f k8s-manifests/mssql-deployment.yaml
|
||||
kubectl apply -f k8s-manifests/gitea-deployment.yaml
|
||||
kubectl apply -f k8s-manifests/nexus-deployment.yaml
|
||||
kubectl apply -f k8s-manifests/mssql-backup-cronjob.yaml
|
||||
```
|
||||
|
||||
### Check Status:
|
||||
```bash
|
||||
kubectl get pods
|
||||
kubectl get pvc
|
||||
kubectl get svc
|
||||
```
|
||||
|
||||
### View Logs:
|
||||
```bash
|
||||
kubectl logs -f deploy/mssql
|
||||
kubectl logs -f deploy/gitea
|
||||
kubectl logs -f deploy/nexus
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔐 Credentials Summary
|
||||
|
||||
| Service | Username | Password | Server |
|
||||
|---------|----------|----------|--------|
|
||||
| Staging SSH | root | 87zH26nbqT | 194.5.195.53 |
|
||||
| Production SSH | root | 87zH26nbqT | 45.149.79.127 |
|
||||
| Staging MSSQL | sa | 87zH26nbqT | mssql-svc:1433 |
|
||||
| Production MSSQL | sa | YourStrong@Passw0rd | 45.149.79.127:31433 |
|
||||
| Gitea | admin | (set during install) | 194.5.195.53 |
|
||||
|
||||
---
|
||||
|
||||
## 📋 Troubleshooting
|
||||
|
||||
### MSSQL Not Starting:
|
||||
1. Check if volumeMounts exists in deployment
|
||||
2. Verify password matches stored in database
|
||||
3. Use single-user mode to reset password if needed
|
||||
|
||||
### Gitea Shows Install Page:
|
||||
1. Check if volumeMounts exists (must mount to `/data`)
|
||||
2. Verify MSSQL is running and accessible
|
||||
3. Check `/data/gitea/conf/app.ini` for database config
|
||||
|
||||
### Images Not Pulling:
|
||||
1. Ensure Nexus is running
|
||||
2. For K3s, add to `/etc/rancher/k3s/registries.yaml`:
|
||||
```yaml
|
||||
mirrors:
|
||||
"194.5.195.53:32082":
|
||||
endpoint:
|
||||
- "http://194.5.195.53:32082"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📦 Images in Nexus Registry
|
||||
|
||||
| Image | Tag | Purpose |
|
||||
|-------|-----|---------|
|
||||
| `gitea/gitea` | `1.25.3`, `latest` | Git server |
|
||||
| `gitea/act_runner` | `0.2.11`, `latest` | CI/CD runner |
|
||||
| `mcr.microsoft.com/mssql/server` | `2022-CU16` | Database |
|
||||
| `registry.k8s.io/ingress-nginx/controller` | `v1.14.1` | Ingress |
|
||||
|
||||
List all images:
|
||||
```bash
|
||||
curl -s http://194.5.195.53:32082/v2/_catalog
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
## 🔧 CMS Ingress & Kestrel Protocol (بروزشده: February 2026)
|
||||
|
||||
### تنظیمات Kestrel:
|
||||
| متغیر | مقدار قبلی | مقدار فعلی |
|
||||
|--------|-----------|------------|
|
||||
| `Kestrel__EndpointDefaults__Protocols` | `Http2` | `Http1AndHttp2` |
|
||||
|
||||
> با `Http1AndHttp2` هم gRPC (HTTP/2) و هم REST/HTTP (HTTP/1.1) روی یک پورت کار میکنن.
|
||||
|
||||
### تنظیمات Ingress CMS:
|
||||
| Annotation | مقدار قبلی | مقدار فعلی |
|
||||
|-----------|-----------|------------|
|
||||
| `backend-protocol` | `GRPC` | حذف شد |
|
||||
| `grpc-backend` | `true` | حذف شد |
|
||||
| `ssl-redirect` | `true` | `true` |
|
||||
| `cert-manager.io/cluster-issuer` | `letsencrypt-prod` | `letsencrypt-prod` |
|
||||
|
||||
> ⚠️ FrontOffice از gRPC-Web استفاده میکنه که روی HTTP/1.1 هم کار میکنه — نیازی به annotation GRPC نیست.
|
||||
|
||||
### NuGet Package (Proto):
|
||||
| پکیج | نسخه | رجیستری |
|
||||
|-------|-------|--------|
|
||||
| `Foursat.CMSMicroservice.Protobuf` | `0.0.179` | Nexus (`foursat-nuget-hosted`) |
|
||||
|
||||
### Gitea Secrets (هر ۳ ریپو):
|
||||
| Secret | CMS | FrontOffice | BackOffice |
|
||||
|--------|-----|-------------|------------|
|
||||
| `REGISTRY_PASSWORD` | ✅ | ✅ | ✅ |
|
||||
| `SERVER_PASSWORD` | ✅ | ✅ | ✅ |
|
||||
| `KUBECONFIG` | ✅ | ✅ | ✅ |
|
||||
|
||||
---
|
||||
|
||||
*Last Updated: February 17, 2026*
|
||||
|
||||
|
||||
---
|
||||
|
||||
# وضعیت استقرار فعلی
|
||||
|
||||
# ✅ FourSat Offline Deployment - Complete Status
|
||||
|
||||
## 📦 Available Package & Image Repositories
|
||||
|
||||
### 1. Docker Registry (Primary - Already Working)
|
||||
**Location:** `194.5.195.53:32500`
|
||||
**Status:** ✅ **Active & Working**
|
||||
**Purpose:** Docker image caching for Kubernetes
|
||||
|
||||
**Cached Images:**
|
||||
```
|
||||
✅ nginx:alpine → localhost:32500/nginx:alpine
|
||||
✅ dotnet/aspnet:9.0 → localhost:32500/dotnet/aspnet:9.0
|
||||
✅ dotnet/sdk:9.0 → localhost:32500/dotnet/sdk:9.0
|
||||
```
|
||||
|
||||
**Storage:** 881MB in `/var/lib/registry`
|
||||
|
||||
**Usage:**
|
||||
```bash
|
||||
# Pull from local registry
|
||||
crictl pull 194.5.195.53:32500/nginx:alpine
|
||||
crictl pull 194.5.195.53:32500/dotnet/aspnet:9.0
|
||||
crictl pull 194.5.195.53:32500/dotnet/sdk:9.0
|
||||
|
||||
# Or with docker
|
||||
docker pull 194.5.195.53:32500/nginx:alpine
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 2. Nexus Repository Manager (Newly Deployed)
|
||||
**Location:** `https://nexus.se.kbs1.ir` (194.5.195.53:32081)
|
||||
**Status:** ✅ **Active & Configured**
|
||||
**Purpose:** NuGet package caching + Docker images (future)
|
||||
|
||||
#### NuGet Repositories (✅ Ready)
|
||||
- **nuget-all** (Group) - https://nexus.se.kbs1.ir/repository/nuget-all/index.json
|
||||
- Combines: nuget-org-proxy + foursat-nuget-hosted
|
||||
- **Use this in all projects** ← Already configured!
|
||||
|
||||
- **nuget-org-proxy** (Proxy) - Caches packages from nuget.org
|
||||
- **foursat-nuget-hosted** (Hosted) - For private packages
|
||||
|
||||
#### Docker Repositories (🚧 Configured but not yet populated)
|
||||
- **docker-all** (Group) - Port 32084
|
||||
- Combines: docker-hosted + docker-hub-proxy
|
||||
|
||||
- **docker-hosted** (Hosted) - Port 32082
|
||||
- **docker-hub-proxy** (Proxy) - Port 32083
|
||||
|
||||
**Note:** Docker registry ports in Nexus are not yet externally accessible. Currently using the standalone Docker Registry (32500) instead.
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Current Configuration
|
||||
|
||||
### Projects Using Nexus for NuGet
|
||||
All NuGet.config files updated to use Nexus as primary source:
|
||||
|
||||
```xml
|
||||
<packageSources>
|
||||
<clear />
|
||||
<add key="Nexus" value="https://nexus.se.kbs1.ir/repository/nuget-all/index.json" />
|
||||
<!-- Fallback: Direct Gitea -->
|
||||
<add key="FourSat" value="https://git.afrino.co/api/packages/FourSat/nuget/index.json" />
|
||||
<add key="Afrino" value="https://git.afrino.co/api/packages/Afrino/nuget/index.json" />
|
||||
</packageSources>
|
||||
```
|
||||
|
||||
**Updated files:**
|
||||
- ✅ BackOffice/src/BackOffice/NuGet.config
|
||||
- ✅ BackOffice.BFF/src/BackOffice.BFF.WebApi/NuGet.config
|
||||
- ✅ FrontOffice/src/FrontOffice.Main/NuGet.config
|
||||
- ✅ FrontOffice.BFF/src/FrontOffice.BFF.WebApi/NuGet.config
|
||||
|
||||
### Dockerfiles Using Local Registry
|
||||
All Dockerfiles updated to pull from local registry:
|
||||
|
||||
```dockerfile
|
||||
# Before
|
||||
FROM mcr.microsoft.com/dotnet/aspnet:9.0
|
||||
|
||||
# After
|
||||
FROM 194.5.195.53:32500/dotnet/aspnet:9.0
|
||||
```
|
||||
|
||||
**Updated files:**
|
||||
- ✅ BackOffice/src/BackOffice/Dockerfile
|
||||
- ✅ BackOffice.BFF/src/BackOffice.BFF.WebApi/Dockerfile
|
||||
- ✅ FrontOffice/src/FrontOffice.Main/Dockerfile
|
||||
- ✅ FrontOffice.BFF/src/FrontOffice.BFF.WebApi/Dockerfile
|
||||
- ✅ CMS/Dockerfile
|
||||
|
||||
### Workflows Using Insecure Registry
|
||||
All Gitea Actions workflows configured for local registry:
|
||||
|
||||
```yaml
|
||||
jobs:
|
||||
build:
|
||||
container:
|
||||
image: 194.5.195.53:32500/dotnet/sdk:9.0
|
||||
options: --add-host=host.docker.internal:host-gateway
|
||||
```
|
||||
|
||||
**Updated files:**
|
||||
- ✅ .gitea/workflows/backoffice-build.yml
|
||||
- ✅ .gitea/workflows/backoffice-bff-build.yml
|
||||
- ✅ .gitea/workflows/frontoffice-build.yml
|
||||
- ✅ .gitea/workflows/frontoffice-bff-build.yml
|
||||
- ✅ .gitea/workflows/cms-build.yml
|
||||
|
||||
---
|
||||
|
||||
## 🚀 How It Works
|
||||
|
||||
### NuGet Package Workflow
|
||||
1. **First restore:** `dotnet restore`
|
||||
- Downloads packages from nuget.org **via Nexus proxy**
|
||||
- Nexus caches packages locally
|
||||
|
||||
2. **Subsequent restores:**
|
||||
- Served from Nexus cache
|
||||
- **No internet required!** ✅
|
||||
|
||||
### Docker Image Workflow
|
||||
1. **Build time:**
|
||||
```dockerfile
|
||||
FROM 194.5.195.53:32500/dotnet/aspnet:9.0
|
||||
```
|
||||
- Pulls from local Docker Registry
|
||||
- **No internet required!** ✅
|
||||
|
||||
2. **Runtime (Kubernetes):**
|
||||
```yaml
|
||||
image: 194.5.195.53:32500/nginx:alpine
|
||||
```
|
||||
- Pulls from local registry
|
||||
- **No internet required!** ✅
|
||||
|
||||
---
|
||||
|
||||
## 📊 Storage Usage
|
||||
|
||||
| Service | Storage Path | Size | Purpose |
|
||||
|---------|--------------|------|---------|
|
||||
| Docker Registry | `/var/lib/registry` | 881 MB | Cached Docker images |
|
||||
| Nexus | `/var/lib/nexus` | ~700 MB | NuGet packages + metadata |
|
||||
| Containerd | `/var/lib/containerd` | ~2.4 GB | K8s runtime images |
|
||||
|
||||
**Total offline assets:** ~4 GB
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Benefits Achieved
|
||||
|
||||
### ✅ Complete Offline Capability
|
||||
- Docker images cached locally
|
||||
- NuGet packages cached after first download
|
||||
- No repeated downloads from internet
|
||||
- Faster builds and deployments
|
||||
|
||||
### ✅ Bandwidth Savings
|
||||
- Each dotnet/sdk:9.0 pull: 859 MB saved
|
||||
- Each dotnet/aspnet:9.0 pull: 227 MB saved
|
||||
- Each NuGet package: downloaded once, cached forever
|
||||
|
||||
### ✅ Build Speed Improvements
|
||||
- Local registry: ~10x faster than Docker Hub
|
||||
- Cached NuGet packages: ~5x faster restores
|
||||
- CI/CD builds complete in minutes, not hours
|
||||
|
||||
### ✅ Reliability
|
||||
- No dependency on external services
|
||||
- Works even when internet is down
|
||||
- Consistent build environment
|
||||
|
||||
---
|
||||
|
||||
## 🔍 Verification Commands
|
||||
|
||||
### Check Docker Registry
|
||||
```bash
|
||||
# List images in registry
|
||||
curl -s http://194.5.195.53:32500/v2/_catalog | python3 -m json.tool
|
||||
|
||||
# Check storage
|
||||
ssh root@194.5.195.53 "du -sh /var/lib/registry"
|
||||
```
|
||||
|
||||
### Check Nexus NuGet
|
||||
```bash
|
||||
# Test NuGet connectivity
|
||||
dotnet nuget list source
|
||||
|
||||
# Test package download
|
||||
dotnet add package Newtonsoft.Json
|
||||
```
|
||||
|
||||
### Check Nexus UI
|
||||
```bash
|
||||
# Open in browser
|
||||
https://nexus.se.kbs1.ir
|
||||
|
||||
# Login: admin / 87zH26nbqT
|
||||
# Browse → docker-hosted (for future Docker images)
|
||||
# Browse → nuget-org-proxy (for cached NuGet packages)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ Maintenance
|
||||
|
||||
### Add New Docker Image to Local Registry
|
||||
```bash
|
||||
# On server with internet (172.19.101.100)
|
||||
docker pull <new-image>
|
||||
docker save <new-image> -o /tmp/new-image.tar
|
||||
|
||||
# Transfer to main server
|
||||
scp /tmp/new-image.tar root@194.5.195.53:/tmp/
|
||||
|
||||
# On main server (194.5.195.53)
|
||||
ctr -n k8s.io images import /tmp/new-image.tar
|
||||
ctr -n k8s.io images tag <new-image> 194.5.195.53:32500/<new-image>
|
||||
ctr -n k8s.io images push --plain-http 194.5.195.53:32500/<new-image>
|
||||
```
|
||||
|
||||
### Clear NuGet Cache (if needed)
|
||||
```bash
|
||||
# Via Nexus UI
|
||||
Settings → Repository → Repositories → nuget-org-proxy → Repair - Invalidate cache
|
||||
|
||||
# Or delete and recreate repository
|
||||
```
|
||||
|
||||
### Backup Cached Assets
|
||||
```bash
|
||||
# Docker Registry
|
||||
tar -czf docker-registry-backup.tar.gz /var/lib/registry/
|
||||
|
||||
# Nexus
|
||||
kubectl scale deployment nexus --replicas=0
|
||||
tar -czf nexus-backup.tar.gz /var/lib/nexus/
|
||||
kubectl scale deployment nexus --replicas=1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📝 Files Created/Modified
|
||||
|
||||
### Deployment Files
|
||||
- ✅ `deployment/docker-registry-k8s.yaml` - Docker Registry deployment
|
||||
- ✅ `deployment/nexus-k8s.yaml` - Nexus deployment
|
||||
- ✅ `deployment/nexus-ingress.yaml` - Nexus Ingress with TLS
|
||||
- ✅ `deployment/create-nexus-repos.sh` - Repository creation script
|
||||
- ✅ `deployment/NEXUS-COMPLETE-SETUP.md` - Nexus setup guide
|
||||
- ✅ `deployment/COMPLETE-SETUP-DOCUMENTATION.md` - Full journey documentation
|
||||
- ✅ `deployment/DEPLOYMENT-STATUS.md` - This file
|
||||
|
||||
### Configuration Files
|
||||
- ✅ 4x NuGet.config files (all projects)
|
||||
- ✅ 5x Dockerfile files (all services)
|
||||
- ✅ 5x Gitea workflow files (all pipelines)
|
||||
|
||||
---
|
||||
|
||||
## 🎉 Summary
|
||||
|
||||
**Status:** ✅ **Fully Operational**
|
||||
|
||||
You now have:
|
||||
1. ✅ **Local Docker Registry** caching all base images
|
||||
2. ✅ **Nexus** caching all NuGet packages
|
||||
3. ✅ **All projects configured** to use local sources
|
||||
4. ✅ **Complete offline deployment capability**
|
||||
|
||||
**Next steps:**
|
||||
- Test a full build: `dotnet restore && dotnet build`
|
||||
- Deploy a service: Images will pull from local registry
|
||||
- Monitor Nexus: Watch NuGet packages cache on first restore
|
||||
|
||||
**Result:** Zero downloads required after initial cache population! 🚀
|
||||
@@ -1,82 +0,0 @@
|
||||
# ⚠️ CRITICAL WARNING: ingress-nginx with K3s
|
||||
|
||||
## The Problem
|
||||
|
||||
When using **K3s** with the built-in **svclb (ServiceLB)**, DO NOT add `hostNetwork: true` to the ingress-nginx controller.
|
||||
|
||||
## Why This Happens
|
||||
|
||||
K3s automatically deploys `svclb-*` pods when you create a `LoadBalancer` service. These svclb pods:
|
||||
- Use `hostNetwork: true` by design
|
||||
- Bind to ports 80 and 443 on the host
|
||||
|
||||
If you also add `hostNetwork: true` to ingress-nginx-controller:
|
||||
- **Both** svclb pods AND ingress-nginx pods try to bind to ports 80/443
|
||||
- This causes bind conflicts
|
||||
- External traffic cannot reach the ingress controller
|
||||
- You'll see "connection refused" or routing failures
|
||||
|
||||
## The Solution
|
||||
|
||||
**Remove `hostNetwork: true`** from ingress-nginx-controller DaemonSet/Deployment.
|
||||
|
||||
```bash
|
||||
# Check current config
|
||||
kubectl get ds -n ingress-nginx ingress-nginx-controller -o yaml | grep -A5 hostNetwork
|
||||
|
||||
# If hostNetwork is true, patch to remove it:
|
||||
kubectl patch ds -n ingress-nginx ingress-nginx-controller --type='json' -p='[{"op":"remove","path":"/spec/template/spec/hostNetwork"}]'
|
||||
|
||||
# Restart pods
|
||||
kubectl rollout restart ds -n ingress-nginx ingress-nginx-controller
|
||||
```
|
||||
|
||||
## How K3s svclb Works
|
||||
|
||||
```
|
||||
External Request (port 80/443)
|
||||
│
|
||||
▼
|
||||
┌───────────────────┐
|
||||
│ svclb-* pod │ ← hostNetwork: true, binds to 80/443
|
||||
│ (K3s ServiceLB) │
|
||||
└─────────┬─────────┘
|
||||
│
|
||||
▼ forwards to service
|
||||
┌───────────────────────────────┐
|
||||
│ ingress-nginx-controller svc │ (LoadBalancer type)
|
||||
│ ClusterIP:10.43.x.x:80/443 │
|
||||
└─────────┬─────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌───────────────────────────────┐
|
||||
│ ingress-nginx-controller pod │ ← NO hostNetwork needed
|
||||
│ Listens on container ports │
|
||||
└───────────────────────────────┘
|
||||
```
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
# Check svclb pods are running
|
||||
kubectl get pods -A | grep svclb
|
||||
|
||||
# Should see:
|
||||
# kube-system svclb-ingress-nginx-controller-xxxxx Running
|
||||
|
||||
# Verify ports are accessible
|
||||
curl -I http://SERVER_IP
|
||||
|
||||
# Should get HTTP response from ingress-nginx
|
||||
```
|
||||
|
||||
## Related Issues
|
||||
|
||||
- If you use `NodePort` instead of `LoadBalancer`, svclb pods won't be created
|
||||
- If you disable K3s ServiceLB and use MetalLB, different rules apply
|
||||
- Cloud providers with real LoadBalancers also don't need hostNetwork
|
||||
|
||||
---
|
||||
|
||||
*Date: 2025-01-18*
|
||||
*Issue discovered while deploying FourSat infrastructure*
|
||||
@@ -1,832 +0,0 @@
|
||||
# راهنمای دیپلوی آفلاین FourSat
|
||||
|
||||
> تاریخ: 2026-01-29
|
||||
> هدف: دیپلوی بدون نیاز به اینترنت خارجی
|
||||
|
||||
---
|
||||
|
||||
## 📋 خلاصه اجرایی
|
||||
|
||||
این راهنما شامل تنظیمات لازم برای دیپلوی کامل آفلاین پروژه FourSat است. با استفاده از Nexus به عنوان registry مرکزی و mirror های ایرانی به عنوان fallback، نیازی به اینترنت خارجی نیست.
|
||||
|
||||
---
|
||||
|
||||
## 🖥️ سرورها
|
||||
|
||||
| سرور | IP | نقش | رمز عبور |
|
||||
|------|-----|------|----------|
|
||||
| **Stage** | `194.5.195.53` | Nexus, Gitea, Runner | `87zH26nbqT` |
|
||||
| **Production** | `45.149.79.127` | K8S Production | `87zH26nbqT` |
|
||||
|
||||
---
|
||||
|
||||
## 🐳 Nexus Registry
|
||||
|
||||
### پورتها
|
||||
| سرویس | پورت | پروتکل |
|
||||
|--------|------|--------|
|
||||
| Nexus UI | `32081` | HTTP |
|
||||
| Docker Registry | `32082` | HTTP (insecure) |
|
||||
| NuGet | `32081/repository/nuget-group/index.json` | HTTP |
|
||||
|
||||
### Credentials
|
||||
```
|
||||
Username: admin
|
||||
Password: 87zH26nbqT
|
||||
```
|
||||
|
||||
### ریپوزیتوریهای Docker
|
||||
| نام | نوع | توضیح |
|
||||
|-----|------|-------|
|
||||
| `docker-hosted` | hosted | ایمیجهای پروژه |
|
||||
| `docker-hub-proxy` | proxy | پروکسی Docker Hub |
|
||||
| `docker-arvancloud-proxy` | proxy | پروکسی ArvanCloud |
|
||||
| `docker-all` | group | گروه همه ریپوها |
|
||||
|
||||
### ریپوزیتوریهای NuGet
|
||||
| نام | نوع | توضیح |
|
||||
|-----|------|-------|
|
||||
| `foursat-nuget-hosted` | hosted | پکیجهای پروتوباف |
|
||||
| `nuget.org-proxy` | proxy | پروکسی NuGet.org |
|
||||
| `nuget-runflare-proxy` | proxy | پروکسی Runflare |
|
||||
| `nuget-group` | group | گروه همه ریپوها |
|
||||
|
||||
---
|
||||
|
||||
## 🪞 Mirror های ایرانی (Fallback)
|
||||
|
||||
### Docker
|
||||
```
|
||||
https://docker.arvancloud.ir
|
||||
```
|
||||
|
||||
### APT/Ubuntu
|
||||
```
|
||||
http://mirror.arvancloud.ir/ubuntu
|
||||
```
|
||||
|
||||
### NuGet
|
||||
```
|
||||
https://mirror-nuget.runflare.com/v3/index.json
|
||||
```
|
||||
|
||||
### PyPI
|
||||
```
|
||||
https://mirror-pypi.runflare.com/simple
|
||||
```
|
||||
|
||||
### NPM
|
||||
```
|
||||
https://mirror-npm.runflare.com
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📦 ایمیجهای ذخیره شده در Nexus
|
||||
|
||||
| ایمیج | تگ | سایز تقریبی |
|
||||
|-------|-----|-------------|
|
||||
| `gitea/gitea` | `1.25.3` | ~78MB |
|
||||
| `mcr.microsoft.com/mssql/server` | `2022-CU16-ubuntu-22.04` | ~1.6GB |
|
||||
| `gitea/act_runner` | `0.2.11`, `latest` | ~50MB |
|
||||
| `registry.k8s.io/ingress-nginx/controller` | `v1.14.1` | ~280MB |
|
||||
| `dotnet/sdk` | `9.0` | ~900MB |
|
||||
| `dotnet/aspnet` | `9.0` | ~220MB |
|
||||
| `library/nginx` | `alpine` | ~40MB |
|
||||
| `docker` | `dind` | ~400MB |
|
||||
| `docker-sshpass` | `latest` | ~500MB |
|
||||
|
||||
---
|
||||
|
||||
## ⚙️ تنظیمات K3s
|
||||
|
||||
### فایل: `/etc/rancher/k3s/registries.yaml`
|
||||
|
||||
```yaml
|
||||
# Registry Mirrors Configuration
|
||||
# Primary: Nexus (194.5.195.53:32082)
|
||||
# Fallback: ArvanCloud (docker.arvancloud.ir)
|
||||
|
||||
mirrors:
|
||||
"docker.io":
|
||||
endpoint:
|
||||
- "http://194.5.195.53:32082"
|
||||
- "https://docker.arvancloud.ir"
|
||||
- "https://registry-1.docker.io"
|
||||
"194.5.195.53:32082":
|
||||
endpoint:
|
||||
- "http://194.5.195.53:32082"
|
||||
"ghcr.io":
|
||||
endpoint:
|
||||
- "http://194.5.195.53:32082"
|
||||
- "https://docker.arvancloud.ir"
|
||||
"gcr.io":
|
||||
endpoint:
|
||||
- "http://194.5.195.53:32082"
|
||||
- "https://docker.arvancloud.ir"
|
||||
"registry.k8s.io":
|
||||
endpoint:
|
||||
- "http://194.5.195.53:32082"
|
||||
- "https://docker.arvancloud.ir"
|
||||
"quay.io":
|
||||
endpoint:
|
||||
- "http://194.5.195.53:32082"
|
||||
- "https://docker.arvancloud.ir"
|
||||
"mcr.microsoft.com":
|
||||
endpoint:
|
||||
- "http://194.5.195.53:32082"
|
||||
- "https://docker.arvancloud.ir"
|
||||
|
||||
configs:
|
||||
"194.5.195.53:32082":
|
||||
auth:
|
||||
username: admin
|
||||
password: 87zH26nbqT
|
||||
```
|
||||
|
||||
### اعمال تغییرات
|
||||
```bash
|
||||
sudo systemctl restart k3s
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📝 تنظیمات APT
|
||||
|
||||
### فایل: `/etc/apt/sources.list.d/ubuntu.sources`
|
||||
|
||||
```
|
||||
Types: deb
|
||||
URIs: http://mirror.arvancloud.ir/ubuntu http://archive.ubuntu.com/ubuntu
|
||||
Suites: noble noble-updates noble-backports
|
||||
Components: main restricted universe multiverse
|
||||
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg
|
||||
|
||||
Types: deb
|
||||
URIs: http://mirror.arvancloud.ir/ubuntu http://security.ubuntu.com/ubuntu
|
||||
Suites: noble-security
|
||||
Components: main restricted universe multiverse
|
||||
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🐍 تنظیمات PIP
|
||||
|
||||
### فایل: `/root/.config/pip/pip.conf`
|
||||
|
||||
```ini
|
||||
[global]
|
||||
index-url = https://pypi.org/simple
|
||||
extra-index-url = https://mirror-pypi.runflare.com/simple
|
||||
trusted-host = mirror-pypi.runflare.com
|
||||
timeout = 60
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📦 تنظیمات NPM
|
||||
|
||||
### فایل: `/root/.npmrc`
|
||||
|
||||
```
|
||||
registry=https://registry.npmjs.org/
|
||||
# Fallback (uncomment if needed):
|
||||
# registry=https://mirror-npm.runflare.com
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔧 تنظیمات Gitea Runner
|
||||
|
||||
### مشکل: Runner نمیتونه از Nexus (HTTP) pull کنه
|
||||
|
||||
**علت:** Docker daemon داخل Runner سعی میکنه با HTTPS وصل بشه.
|
||||
|
||||
**راه حل:** ConfigMap برای daemon.json
|
||||
|
||||
### ConfigMap
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: docker-daemon-config
|
||||
data:
|
||||
daemon.json: |
|
||||
{
|
||||
"insecure-registries": ["194.5.195.53:32082", "194.5.195.53:30080"]
|
||||
}
|
||||
```
|
||||
|
||||
### Deployment Patch
|
||||
```bash
|
||||
kubectl patch deployment gitea-runner --type=json -p='[
|
||||
{
|
||||
"op": "add",
|
||||
"path": "/spec/template/spec/volumes/-",
|
||||
"value": {
|
||||
"name": "docker-config",
|
||||
"configMap": {
|
||||
"name": "docker-daemon-config"
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"op": "add",
|
||||
"path": "/spec/template/spec/containers/0/volumeMounts/-",
|
||||
"value": {
|
||||
"name": "docker-config",
|
||||
"mountPath": "/etc/docker/daemon.json",
|
||||
"subPath": "daemon.json"
|
||||
}
|
||||
}
|
||||
]'
|
||||
```
|
||||
|
||||
### بررسی
|
||||
```bash
|
||||
kubectl exec $(kubectl get pods -l app=gitea-runner -o jsonpath='{.items[0].metadata.name}') \
|
||||
-c docker -- docker info | grep -A 5 'Insecure Registries'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📁 ساختار Dockerfile ها
|
||||
|
||||
### الگوی استاندارد (با Nexus)
|
||||
```dockerfile
|
||||
FROM 194.5.195.53:32082/dotnet/sdk:9.0 AS build
|
||||
WORKDIR /src
|
||||
|
||||
# Copy NuGet config
|
||||
COPY src/NuGet.config ./
|
||||
|
||||
# Restore and build
|
||||
RUN dotnet restore "Project.csproj" --configfile NuGet.config
|
||||
RUN dotnet publish "Project.csproj" -c Release -o /app/publish --no-restore
|
||||
|
||||
FROM 194.5.195.53:32082/dotnet/aspnet:9.0 AS runtime
|
||||
WORKDIR /app
|
||||
COPY --from=build /app/publish .
|
||||
ENTRYPOINT ["dotnet", "Project.dll"]
|
||||
```
|
||||
|
||||
### NuGet.config
|
||||
```xml
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<configuration>
|
||||
<packageSources>
|
||||
<clear />
|
||||
<add key="nexus" value="http://194.5.195.53:32081/repository/nuget-group/index.json" />
|
||||
</packageSources>
|
||||
</configuration>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔄 ساختار Workflow (CI/CD)
|
||||
|
||||
### الگوی استاندارد `kub-deploy.yml`
|
||||
```yaml
|
||||
name: Build and Deploy
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- kub-stage # یا production
|
||||
|
||||
env:
|
||||
REGISTRY: 194.5.195.53:30080
|
||||
IMAGE_NAME: admin/project-name
|
||||
K8S_SERVER: 194.5.195.53 # یا 45.149.79.127 برای Production
|
||||
|
||||
jobs:
|
||||
build-and-deploy:
|
||||
runs-on: ubuntu-latest
|
||||
container:
|
||||
image: 194.5.195.53:32082/docker-sshpass:latest
|
||||
options: --privileged
|
||||
steps:
|
||||
- name: Start Docker daemon
|
||||
run: |
|
||||
mkdir -p /etc/docker
|
||||
cat > /etc/docker/daemon.json << 'DAEMON'
|
||||
{
|
||||
"insecure-registries": ["194.5.195.53:30080", "194.5.195.53:32082"]
|
||||
}
|
||||
DAEMON
|
||||
dockerd &
|
||||
for i in $(seq 1 90); do
|
||||
docker info >/dev/null 2>&1 && break || sleep 2
|
||||
done
|
||||
|
||||
- name: Checkout code
|
||||
run: |
|
||||
git clone --depth 1 --branch $BRANCH http://gitea-svc:3000/admin/PROJECT.git .
|
||||
|
||||
- name: Build Docker Image
|
||||
run: |
|
||||
docker build -t ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest .
|
||||
|
||||
- name: Push to Registry
|
||||
run: |
|
||||
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login ${{ env.REGISTRY }} -u admin --password-stdin
|
||||
docker push ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest
|
||||
|
||||
- name: Deploy
|
||||
run: |
|
||||
sshpass -p "${{ secrets.K8S_SSH_PASSWORD }}" ssh -o StrictHostKeyChecking=no root@${{ env.K8S_SERVER }} \
|
||||
"kubectl rollout restart deployment/PROJECT"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔐 Secrets مورد نیاز در Gitea
|
||||
|
||||
| Secret | مقدار | توضیح |
|
||||
|--------|-------|-------|
|
||||
| `REGISTRY_PASSWORD` | `87zH26nbqT` | رمز Gitea Registry |
|
||||
| `K8S_SSH_PASSWORD` | `87zH26nbqT` | رمز SSH سرور |
|
||||
|
||||
---
|
||||
|
||||
## 💾 بکاپ روزانه MSSQL
|
||||
|
||||
### CronJob
|
||||
```yaml
|
||||
apiVersion: batch/v1
|
||||
kind: CronJob
|
||||
metadata:
|
||||
name: mssql-backup
|
||||
spec:
|
||||
schedule: "0 2 * * *" # هر روز ساعت 2 صبح
|
||||
jobTemplate:
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: backup
|
||||
image: 194.5.195.53:32082/mcr.microsoft.com/mssql-tools:latest
|
||||
command:
|
||||
- /bin/bash
|
||||
- -c
|
||||
- |
|
||||
DATE=$(date +%Y%m%d)
|
||||
for DB in gitea Foursat; do
|
||||
/opt/mssql-tools/bin/sqlcmd -S mssql-svc -U sa -P '87zH26nbqT' \
|
||||
-Q "BACKUP DATABASE [$DB] TO DISK='/backups/${DB}_${DATE}.bak'"
|
||||
done
|
||||
volumeMounts:
|
||||
- name: backup-volume
|
||||
mountPath: /backups
|
||||
volumes:
|
||||
- name: backup-volume
|
||||
hostPath:
|
||||
path: /mnt/mssql-backups
|
||||
restartPolicy: OnFailure
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 خلاصه پروژهها
|
||||
|
||||
| پروژه | Dockerfile | Workflow Stage | Workflow Prod |
|
||||
|-------|------------|----------------|---------------|
|
||||
| BackOffice | ✅ Nexus | ✅ | ✅ |
|
||||
| BackOffice.BFF | ✅ Nexus | ✅ | ✅ |
|
||||
| CMS | ✅ Nexus | ✅ | ✅ |
|
||||
| FrontOffice | ✅ Nexus | ✅ | ✅ |
|
||||
| FrontOffice.BFF | ✅ Nexus | ✅ | ✅ |
|
||||
|
||||
---
|
||||
|
||||
## 🚨 Troubleshooting
|
||||
|
||||
### مشکل: Image pull failed - HTTPS error
|
||||
```
|
||||
Error: http: server gave HTTP response to HTTPS client
|
||||
```
|
||||
**راه حل:** اضافه کردن registry به insecure-registries
|
||||
|
||||
### مشکل: NuGet restore failed
|
||||
**راه حل:** بررسی NuGet.config و اتصال به Nexus
|
||||
|
||||
### مشکل: Runner CrashLoopBackOff
|
||||
**راه حل:** بررسی لاگها با `kubectl logs`
|
||||
|
||||
### مشکل: K3s نمیتونه pull کنه
|
||||
**راه حل:** بررسی `/etc/rancher/k3s/registries.yaml` و restart K3s
|
||||
|
||||
---
|
||||
|
||||
## 📞 دستورات مفید
|
||||
|
||||
### بررسی وضعیت Runner
|
||||
```bash
|
||||
kubectl get pods -l app=gitea-runner
|
||||
kubectl logs -l app=gitea-runner -c runner --tail=50
|
||||
```
|
||||
|
||||
### تست pull از Nexus
|
||||
```bash
|
||||
crictl pull 194.5.195.53:32082/dotnet/sdk:9.0
|
||||
```
|
||||
|
||||
### بررسی ایمیجها در Nexus
|
||||
```bash
|
||||
curl -u admin:87zH26nbqT http://194.5.195.53:32082/v2/_catalog
|
||||
```
|
||||
|
||||
### Restart K3s
|
||||
```bash
|
||||
sudo systemctl restart k3s
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📅 تاریخچه تغییرات
|
||||
|
||||
| تاریخ | تغییر |
|
||||
|-------|-------|
|
||||
| 2026-01-29 | راهاندازی اولیه، تنظیم Nexus، Runner، و Mirror ها |
|
||||
| 2026-01-29 | تنظیم Production server برای استفاده از Stage Nexus |
|
||||
| 2026-01-29 | آپدیت Dockerfile ها و Workflow های production |
|
||||
| 2026-01-29 | فیکس insecure registry برای Gitea Runner |
|
||||
|
||||
---
|
||||
|
||||
> 📝 این داکیومنت توسط Copilot تهیه شده و باید با تغییرات پروژه بروزرسانی شود.
|
||||
|
||||
|
||||
---
|
||||
|
||||
# تنظیمات Nexus (جزئیات کامل)
|
||||
|
||||
# ✅ Nexus Repository Manager - Complete Setup
|
||||
|
||||
## 📦 Deployed Services
|
||||
|
||||
### Nexus Repository Manager
|
||||
- **Version:** 3.38.0 (Compatible with x86-64-v1 CPU)
|
||||
- **Web UI:** https://nexus.se.kbs1.ir
|
||||
- **NodePort:** http://194.5.195.53:32081
|
||||
- **Credentials:** admin / 87zH26nbqT
|
||||
|
||||
### Kubernetes Resources
|
||||
```bash
|
||||
# Pod
|
||||
kubectl get pod | grep nexus
|
||||
# nexus-6575454f69-fv29t 1/1 Running
|
||||
|
||||
# Service (NodePort)
|
||||
kubectl get svc nexus
|
||||
# Ports: 8081:32081 (Web UI)
|
||||
# 8082:32082 (Docker Hosted)
|
||||
# 8083:32083 (Docker Proxy)
|
||||
# 8084:32084 (Docker Group)
|
||||
|
||||
# Ingress
|
||||
kubectl get ingress nexus-ingress
|
||||
# Host: nexus.se.kbs1.ir
|
||||
# TLS: Self-signed certificate (via cert-manager)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📦 Repositories Created
|
||||
|
||||
### NuGet Repositories
|
||||
|
||||
1. **nuget-org-proxy** (Proxy)
|
||||
- Proxies: https://api.nuget.org/v3/index.json
|
||||
- Caches packages from nuget.org
|
||||
- URL: https://nexus.se.kbs1.ir/repository/nuget-org-proxy/index.json
|
||||
|
||||
2. **foursat-nuget-hosted** (Hosted)
|
||||
- For private FourSat packages
|
||||
- URL: https://nexus.se.kbs1.ir/repository/foursat-nuget-hosted/index.json
|
||||
|
||||
3. **nuget-all** (Group)
|
||||
- Combines: nuget-org-proxy + foursat-nuget-hosted
|
||||
- **Use this URL in projects**
|
||||
- URL: https://nexus.se.kbs1.ir/repository/nuget-all/index.json
|
||||
|
||||
### Docker Repositories
|
||||
|
||||
1. **docker-hosted** (Hosted)
|
||||
- For private Docker images
|
||||
- Port: 32082
|
||||
- URL: 194.5.195.53:32082
|
||||
|
||||
2. **docker-hub-proxy** (Proxy)
|
||||
- Proxies: https://registry-1.docker.io (Docker Hub)
|
||||
- Caches images from Docker Hub
|
||||
- Port: 32083
|
||||
- URL: 194.5.195.53:32083
|
||||
|
||||
3. **docker-all** (Group)
|
||||
- Combines: docker-hosted + docker-hub-proxy
|
||||
- Port: 32084
|
||||
- **Use this for Kubernetes**
|
||||
- URL: 194.5.195.53:32084
|
||||
|
||||
---
|
||||
|
||||
## 🔧 Project Configuration
|
||||
|
||||
### NuGet.config (Already Updated)
|
||||
|
||||
All projects now use Nexus as primary source:
|
||||
|
||||
```xml
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<configuration>
|
||||
<packageSources>
|
||||
<clear />
|
||||
<!-- Nexus as primary source (proxies nuget.org + caches packages) -->
|
||||
<add key="Nexus" value="https://nexus.se.kbs1.ir/repository/nuget-all/index.json" />
|
||||
<!-- Backup: Direct Gitea registries -->
|
||||
<add key="FourSat" value="https://git.afrino.co/api/packages/FourSat/nuget/index.json" />
|
||||
<add key="Afrino" value="https://git.afrino.co/api/packages/Afrino/nuget/index.json" />
|
||||
</packageSources>
|
||||
<packageSourceCredentials>
|
||||
<Nexus>
|
||||
<add key="Username" value="admin" />
|
||||
<add key="ClearTextPassword" value="87zH26nbqT" />
|
||||
</Nexus>
|
||||
<FourSat>
|
||||
<add key="Username" value="masoud" />
|
||||
<add key="ClearTextPassword" value="87zH26nbqT" />
|
||||
</FourSat>
|
||||
<Afrino>
|
||||
<add key="Username" value="systemuser" />
|
||||
<add key="ClearTextPassword" value="sZSA7PTiv3pUSQZ" />
|
||||
</Afrino>
|
||||
</packageSourceCredentials>
|
||||
</configuration>
|
||||
```
|
||||
|
||||
**Updated files:**
|
||||
- ✅ `/BackOffice/src/BackOffice/NuGet.config`
|
||||
- ✅ `/BackOffice.BFF/src/BackOffice.BFF.WebApi/NuGet.config`
|
||||
- ✅ `/FrontOffice/src/FrontOffice.Main/NuGet.config`
|
||||
- ✅ `/FrontOffice.BFF/src/FrontOffice.BFF.WebApi/NuGet.config`
|
||||
|
||||
---
|
||||
|
||||
## 🐳 Docker Registry Configuration
|
||||
|
||||
### For Kubernetes Deployments
|
||||
|
||||
Update `/etc/containerd/config.toml` on all nodes:
|
||||
|
||||
```toml
|
||||
[plugins."io.containerd.grpc.v1.cri".registry]
|
||||
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
|
||||
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."194.5.195.53:32084"]
|
||||
endpoint = ["http://194.5.195.53:32084"]
|
||||
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
|
||||
endpoint = ["http://194.5.195.53:32084"]
|
||||
|
||||
[plugins."io.containerd.grpc.v1.cri".registry.configs]
|
||||
[plugins."io.containerd.grpc.v1.cri".registry.configs."194.5.195.53:32084".auth]
|
||||
username = "admin"
|
||||
password = "87zH26nbqT"
|
||||
```
|
||||
|
||||
Then restart containerd:
|
||||
```bash
|
||||
systemctl restart containerd
|
||||
```
|
||||
|
||||
### For Docker
|
||||
|
||||
Add to `/etc/docker/daemon.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"insecure-registries": [
|
||||
"194.5.195.53:32082",
|
||||
"194.5.195.53:32083",
|
||||
"194.5.195.53:32084"
|
||||
],
|
||||
"registry-mirrors": [
|
||||
"http://194.5.195.53:32084"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Then restart Docker:
|
||||
```bash
|
||||
systemctl restart docker
|
||||
```
|
||||
|
||||
### Docker Login
|
||||
|
||||
```bash
|
||||
docker login 194.5.195.53:32084 -u admin -p 87zH26nbqT
|
||||
docker login 194.5.195.53:32082 -u admin -p 87zH26nbqT
|
||||
docker login 194.5.195.53:32083 -u admin -p 87zH26nbqT
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Usage Examples
|
||||
|
||||
### Pull Docker Images via Nexus Proxy
|
||||
|
||||
```bash
|
||||
# Instead of: docker pull nginx:alpine
|
||||
docker pull 194.5.195.53:32084/nginx:alpine
|
||||
|
||||
# Instead of: docker pull mcr.microsoft.com/dotnet/aspnet:9.0
|
||||
docker pull 194.5.195.53:32084/mcr.microsoft.com/dotnet/aspnet:9.0
|
||||
```
|
||||
|
||||
**First pull:** Downloads from Docker Hub and caches in Nexus
|
||||
**Subsequent pulls:** Served from Nexus cache (no internet needed)
|
||||
|
||||
### Push Private Docker Images
|
||||
|
||||
```bash
|
||||
# Tag image
|
||||
docker tag myapp:latest 194.5.195.53:32082/myapp:latest
|
||||
|
||||
# Push to hosted repository
|
||||
docker push 194.5.195.53:32082/myapp:latest
|
||||
```
|
||||
|
||||
### NuGet Package Restore
|
||||
|
||||
```bash
|
||||
cd /path/to/project
|
||||
dotnet restore
|
||||
```
|
||||
|
||||
**First restore:** Downloads from nuget.org via Nexus proxy
|
||||
**Subsequent restores:** Served from Nexus cache (no internet needed)
|
||||
|
||||
### Publish Private NuGet Packages
|
||||
|
||||
```bash
|
||||
# Pack project
|
||||
dotnet pack MyProject.csproj -c Release
|
||||
|
||||
# Push to Nexus hosted repository
|
||||
dotnet nuget push MyProject.1.0.0.nupkg \
|
||||
--source https://nexus.se.kbs1.ir/repository/foursat-nuget-hosted/ \
|
||||
--api-key admin:87zH26nbqT
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 🔍 Verification
|
||||
|
||||
### Check NuGet Sources
|
||||
|
||||
```bash
|
||||
dotnet nuget list source
|
||||
```
|
||||
|
||||
Expected output:
|
||||
```
|
||||
Registered Sources:
|
||||
1. Nexus [Enabled]
|
||||
https://nexus.se.kbs1.ir/repository/nuget-all/index.json
|
||||
2. FourSat [Enabled]
|
||||
https://git.afrino.co/api/packages/FourSat/nuget/index.json
|
||||
3. Afrino [Enabled]
|
||||
https://git.afrino.co/api/packages/Afrino/nuget/index.json
|
||||
```
|
||||
|
||||
### Test Package Download
|
||||
|
||||
```bash
|
||||
# This should use Nexus as primary source
|
||||
dotnet add package Newtonsoft.Json
|
||||
|
||||
# Check Nexus logs
|
||||
kubectl logs nexus-6575454f69-fv29t | tail -20
|
||||
```
|
||||
|
||||
### Check Cached Packages in Nexus
|
||||
|
||||
```bash
|
||||
# SSH to server
|
||||
ssh root@194.5.195.53
|
||||
|
||||
# Check blob storage
|
||||
du -sh /var/lib/nexus/blobs/default/content/*
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📊 Benefits
|
||||
|
||||
### NuGet Caching
|
||||
- ✅ Packages download once, cached forever
|
||||
- ✅ No repeated downloads from nuget.org
|
||||
- ✅ Faster CI/CD builds
|
||||
- ✅ Works offline after first download
|
||||
|
||||
### Docker Caching
|
||||
- ✅ Base images cached locally (aspnet, sdk, nginx, etc.)
|
||||
- ✅ No repeated downloads from Docker Hub
|
||||
- ✅ Faster Kubernetes deployments
|
||||
- ✅ Works offline after first pull
|
||||
|
||||
### Private Package Hosting
|
||||
- ✅ Host private NuGet packages
|
||||
- ✅ Host private Docker images
|
||||
- ✅ Version control for artifacts
|
||||
- ✅ Access control via credentials
|
||||
|
||||
---
|
||||
|
||||
## 🛠️ Maintenance
|
||||
|
||||
### Check Repository Storage
|
||||
|
||||
Via UI:
|
||||
1. Login to https://nexus.se.kbs1.ir
|
||||
2. Go to: ⚙️ Settings → System → Blob Stores
|
||||
3. View: Storage usage per blob store
|
||||
|
||||
Via API:
|
||||
```bash
|
||||
curl -u admin:87zH26nbqT \
|
||||
http://194.5.195.53:32081/service/rest/v1/blobstores
|
||||
```
|
||||
|
||||
### Clear Cache (if needed)
|
||||
|
||||
Via UI:
|
||||
1. Go to: ⚙️ Settings → Repository → Repositories
|
||||
2. Select repository (e.g., `nuget-org-proxy`)
|
||||
3. Click: **Delete cache**
|
||||
|
||||
### Backup Nexus Data
|
||||
|
||||
```bash
|
||||
# Stop Nexus
|
||||
kubectl scale deployment nexus --replicas=0
|
||||
|
||||
# Backup data
|
||||
tar -czf nexus-backup-$(date +%Y%m%d).tar.gz /var/lib/nexus/
|
||||
|
||||
# Start Nexus
|
||||
kubectl scale deployment nexus --replicas=1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 📝 Files Created
|
||||
|
||||
- ✅ `/deployment/nexus-k8s.yaml` - Kubernetes deployment
|
||||
- ✅ `/deployment/nexus-ingress.yaml` - Ingress with TLS
|
||||
- ✅ `/deployment/create-nexus-repos.sh` - Repository creation script
|
||||
- ✅ `/deployment/NEXUS-COMPLETE-SETUP.md` - This document
|
||||
|
||||
---
|
||||
|
||||
## 🎯 Next Steps
|
||||
|
||||
1. **Test NuGet Caching:**
|
||||
```bash
|
||||
cd BackOffice/src
|
||||
dotnet clean
|
||||
rm -rf ~/.nuget/packages
|
||||
dotnet restore
|
||||
# Check Nexus UI → Browse → nuget-org-proxy
|
||||
```
|
||||
|
||||
2. **Configure Kubernetes to use Docker proxy:**
|
||||
```bash
|
||||
# Update containerd config (see Docker Registry Configuration above)
|
||||
systemctl restart containerd
|
||||
|
||||
# Pull image via Nexus
|
||||
crictl pull 194.5.195.53:32084/nginx:alpine
|
||||
```
|
||||
|
||||
3. **Update Dockerfiles to use local images:**
|
||||
```dockerfile
|
||||
# Instead of: FROM mcr.microsoft.com/dotnet/aspnet:9.0
|
||||
FROM 194.5.195.53:32084/mcr.microsoft.com/dotnet/aspnet:9.0
|
||||
```
|
||||
|
||||
4. **Update CI/CD workflows:**
|
||||
- Already using local registry: `194.5.195.53:32500`
|
||||
- Can migrate to Nexus Docker registry: `194.5.195.53:32084`
|
||||
|
||||
---
|
||||
|
||||
## ✅ Summary
|
||||
|
||||
**Deployed:** Nexus Repository Manager 3.38.0
|
||||
**Accessible:** https://nexus.se.kbs1.ir (with TLS)
|
||||
**Repositories:** NuGet (proxy, hosted, group) + Docker (proxy, hosted, group)
|
||||
**Projects Updated:** All 4 NuGet.config files now use Nexus as primary source
|
||||
**Status:** Ready for production use
|
||||
|
||||
**Result:** Complete offline deployment capability for both NuGet packages and Docker images! 🎉
|
||||
@@ -1,258 +0,0 @@
|
||||
# Server Mirrors Configuration
|
||||
|
||||
**Staging Server:** 194.5.195.53
|
||||
**Production Server:** 45.149.79.127
|
||||
**Date:** 2026-02-17
|
||||
|
||||
---
|
||||
|
||||
## 1. Docker Registry Mirrors (K3s)
|
||||
|
||||
### Staging — `/etc/rancher/k3s/registries.yaml` (194.5.195.53)
|
||||
|
||||
### ترتیب Pull کردن ایمیجها:
|
||||
1. **Nexus** (194.5.195.53:32082) - لوکال
|
||||
2. **ArvanCloud** (docker.arvancloud.ir) - ایران
|
||||
3. **Original Registry** - اصلی
|
||||
|
||||
### رجیستریهای پیکربندی شده:
|
||||
|
||||
| Registry | Mirrors (به ترتیب اولویت) |
|
||||
|----------|--------------------------|
|
||||
| `docker.io` | Nexus → ArvanCloud → registry-1.docker.io |
|
||||
| `ghcr.io` | Nexus → ArvanCloud |
|
||||
| `gcr.io` | Nexus → ArvanCloud |
|
||||
| `registry.k8s.io` | Nexus → ArvanCloud |
|
||||
| `quay.io` | Nexus → ArvanCloud |
|
||||
| `mcr.microsoft.com` | Nexus → ArvanCloud |
|
||||
|
||||
### کانفیگ فعلی:
|
||||
|
||||
```yaml
|
||||
mirrors:
|
||||
"docker.io":
|
||||
endpoint:
|
||||
- "http://194.5.195.53:32082"
|
||||
- "https://docker.arvancloud.ir"
|
||||
- "https://registry-1.docker.io"
|
||||
"ghcr.io":
|
||||
endpoint:
|
||||
- "http://194.5.195.53:32082"
|
||||
- "https://docker.arvancloud.ir"
|
||||
"gcr.io":
|
||||
endpoint:
|
||||
- "http://194.5.195.53:32082"
|
||||
- "https://docker.arvancloud.ir"
|
||||
"registry.k8s.io":
|
||||
endpoint:
|
||||
- "http://194.5.195.53:32082"
|
||||
- "https://docker.arvancloud.ir"
|
||||
"quay.io":
|
||||
endpoint:
|
||||
- "http://194.5.195.53:32082"
|
||||
- "https://docker.arvancloud.ir"
|
||||
"mcr.microsoft.com":
|
||||
endpoint:
|
||||
- "http://194.5.195.53:32082"
|
||||
- "https://docker.arvancloud.ir"
|
||||
configs:
|
||||
"194.5.195.53:32082":
|
||||
auth:
|
||||
username: admin
|
||||
password: 87zH26nbqT
|
||||
```
|
||||
|
||||
### اعمال تغییرات:
|
||||
```bash
|
||||
systemctl restart k3s
|
||||
```
|
||||
|
||||
### Production — `/etc/rancher/k3s/registries.yaml` (45.149.79.127)
|
||||
|
||||
Production server از staging registry ها pull میکنه:
|
||||
|
||||
```yaml
|
||||
mirrors:
|
||||
"docker.io":
|
||||
endpoint:
|
||||
- "http://194.5.195.53:32082"
|
||||
- "https://docker.arvancloud.ir"
|
||||
- "https://registry-1.docker.io"
|
||||
"194.5.195.53:32082":
|
||||
endpoint:
|
||||
- "http://194.5.195.53:32082"
|
||||
"194.5.195.53:30080":
|
||||
endpoint:
|
||||
- "http://194.5.195.53:30080"
|
||||
"git.foursat.afrino.co":
|
||||
endpoint:
|
||||
- "https://git.foursat.afrino.co"
|
||||
"git.se.kbs1.ir":
|
||||
endpoint:
|
||||
- "https://git.se.kbs1.ir"
|
||||
|
||||
configs:
|
||||
"194.5.195.53:32082":
|
||||
auth:
|
||||
username: admin
|
||||
password: 87zH26nbqT
|
||||
"194.5.195.53:30080":
|
||||
auth:
|
||||
username: admin
|
||||
password: 87zH26nbqT
|
||||
"git.foursat.afrino.co":
|
||||
auth:
|
||||
username: admin
|
||||
password: 87zH26nbqT
|
||||
tls:
|
||||
insecure_skip_verify: true
|
||||
"git.se.kbs1.ir":
|
||||
auth:
|
||||
username: admin
|
||||
password: 87zH26nbqT
|
||||
tls:
|
||||
insecure_skip_verify: true
|
||||
```
|
||||
|
||||
> ⚠️ Production از `194.5.195.53:30080` (Gitea container registry) برای pull ایمیجهای CI/CD استفاده میکنه.
|
||||
|
||||
---
|
||||
|
||||
## 2. APT Package Mirrors (Ubuntu 24.04 Noble)
|
||||
|
||||
فایل: `/etc/apt/sources.list.d/ubuntu.sources`
|
||||
|
||||
### ترتیب دانلود پکیجها:
|
||||
1. **ArvanCloud** (mirror.arvancloud.ir) - ایران
|
||||
2. **Ubuntu Official** (archive.ubuntu.com) - اصلی
|
||||
|
||||
### کانفیگ فعلی:
|
||||
|
||||
```
|
||||
Types: deb
|
||||
URIs: http://mirror.arvancloud.ir/ubuntu http://archive.ubuntu.com/ubuntu
|
||||
Suites: noble noble-updates noble-backports
|
||||
Components: main universe restricted multiverse
|
||||
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg
|
||||
|
||||
Types: deb
|
||||
URIs: http://mirror.arvancloud.ir/ubuntu http://security.ubuntu.com/ubuntu
|
||||
Suites: noble-security
|
||||
Components: main universe restricted multiverse
|
||||
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg
|
||||
```
|
||||
|
||||
### اعمال تغییرات:
|
||||
```bash
|
||||
apt update
|
||||
```
|
||||
|
||||
### بکاپ:
|
||||
```
|
||||
/etc/apt/sources.list.d/ubuntu.sources.bak
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. NPM Mirror (Runflare)
|
||||
|
||||
فایل: `/root/.npmrc`
|
||||
|
||||
### تنظیم فعلی:
|
||||
```
|
||||
registry=https://registry.npmjs.org
|
||||
# Fallback mirrors (use if main is slow)
|
||||
# npm config set registry https://mirror-npm.runflare.com
|
||||
```
|
||||
|
||||
### برای تغییر به میرور ایرانی:
|
||||
```bash
|
||||
npm config set registry https://mirror-npm.runflare.com
|
||||
```
|
||||
|
||||
### برای برگشت به اصلی:
|
||||
```bash
|
||||
npm config set registry https://registry.npmjs.org
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. PIP/PyPI Mirror (Runflare)
|
||||
|
||||
فایل: `/root/.config/pip/pip.conf`
|
||||
|
||||
### تنظیم فعلی (با fallback خودکار):
|
||||
```ini
|
||||
[global]
|
||||
index-url = https://pypi.org/simple
|
||||
extra-index-url = https://mirror-pypi.runflare.com/simple
|
||||
trusted-host = mirror-pypi.runflare.com
|
||||
pypi.org
|
||||
```
|
||||
|
||||
**توضیح:** PIP اول از `pypi.org` میگیره، اگه نبود از `mirror-pypi.runflare.com` میگیره.
|
||||
|
||||
---
|
||||
|
||||
## 5. Nexus Repository Manager
|
||||
|
||||
| Item | Value |
|
||||
|------|-------|
|
||||
| URL | http://194.5.195.53:32082 |
|
||||
| UI | http://194.5.195.53:32081 |
|
||||
| Username | admin |
|
||||
| Password | 87zH26nbqT |
|
||||
|
||||
### Docker Repositories:
|
||||
|
||||
| Name | Type | Remote URL |
|
||||
|------|------|------------|
|
||||
| `docker-hosted` | hosted | - |
|
||||
| `docker-arvancloud-proxy` | proxy | https://docker.arvancloud.ir |
|
||||
| `docker-hub-proxy` | proxy | https://registry-1.docker.io |
|
||||
| `docker-all` | group | hosted → arvancloud → docker-hub |
|
||||
|
||||
### NuGet Repositories:
|
||||
|
||||
| Name | Type | Remote URL |
|
||||
|------|------|------------|
|
||||
| `nuget-hosted` | hosted | - |
|
||||
| `foursat-nuget-hosted` | hosted | - |
|
||||
| `nuget-runflare-proxy` | proxy | https://mirror-nuget.runflare.com/v3/index.json |
|
||||
| `nuget.org-proxy` | proxy | https://api.nuget.org/v3/index.json |
|
||||
| `nuget-group` | group | hosted → runflare → nuget.org |
|
||||
|
||||
### ایمیجهای ذخیره شده با ورژن:
|
||||
|
||||
| Image | Tags |
|
||||
|-------|------|
|
||||
| `mcr.microsoft.com/mssql/server` | `2022-CU16`, `2022-latest` |
|
||||
| `gitea/gitea` | `1.25.3`, `latest` |
|
||||
| `gitea/act_runner` | `0.2.11`, `latest` |
|
||||
| `registry.k8s.io/ingress-nginx/controller` | `v1.14.1` |
|
||||
|
||||
---
|
||||
|
||||
## 6. Iranian Mirror URLs Summary
|
||||
|
||||
| سرویس | URL | استفاده |
|
||||
|-------|-----|---------|
|
||||
| Docker | `https://docker.arvancloud.ir` | K3s + Nexus |
|
||||
| Ubuntu APT | `http://mirror.arvancloud.ir/ubuntu` | apt sources |
|
||||
| NuGet | `https://mirror-nuget.runflare.com/v3/index.json` | Nexus proxy |
|
||||
| NPM | `https://mirror-npm.runflare.com` | npmrc (دستی) |
|
||||
| PyPI | `https://mirror-pypi.runflare.com/simple` | pip.conf (fallback) |
|
||||
|
||||
---
|
||||
|
||||
## 7. مزایای این پیکربندی
|
||||
|
||||
✅ **سرعت بالا** - میرورهای ایرانی سریعترن
|
||||
✅ **Fallback خودکار** - اگه میرور در دسترس نبود، اصلی استفاده میشه
|
||||
✅ **Offline Support** - ایمیجهای مهم در Nexus لوکال هستن
|
||||
✅ **کاهش ترافیک خارجی** - اول از سرورهای داخلی استفاده میشه
|
||||
✅ **Cache در Nexus** - پکیجها و ایمیجها cache میشن
|
||||
|
||||
---
|
||||
|
||||
*Last Updated: 2026-01-29*
|
||||
Reference in New Issue
Block a user