Refactor code structure for improved readability and maintainability

This commit is contained in:
masoodafar-web
2026-02-10 22:06:46 +03:30
parent 8f02cec22f
commit 5149b9a89c
182 changed files with 7305 additions and 243980 deletions
+832
View File
@@ -0,0 +1,832 @@
# راهنمای دیپلوی آفلاین FourSat
> تاریخ: 2026-01-29
> هدف: دیپلوی بدون نیاز به اینترنت خارجی
---
## 📋 خلاصه اجرایی
این راهنما شامل تنظیمات لازم برای دیپلوی کامل آفلاین پروژه FourSat است. با استفاده از Nexus به عنوان registry مرکزی و mirror های ایرانی به عنوان fallback، نیازی به اینترنت خارجی نیست.
---
## 🖥️ سرورها
| سرور | IP | نقش | رمز عبور |
|------|-----|------|----------|
| **Stage** | `194.5.195.53` | Nexus, Gitea, Runner | `87zH26nbqT` |
| **Production** | `45.149.79.127` | K8S Production | `87zH26nbqT` |
---
## 🐳 Nexus Registry
### پورت‌ها
| سرویس | پورت | پروتکل |
|--------|------|--------|
| Nexus UI | `32081` | HTTP |
| Docker Registry | `32082` | HTTP (insecure) |
| NuGet | `32081/repository/nuget-group/index.json` | HTTP |
### Credentials
```
Username: admin
Password: 87zH26nbqT
```
### ریپوزیتوری‌های Docker
| نام | نوع | توضیح |
|-----|------|-------|
| `docker-hosted` | hosted | ایمیج‌های پروژه |
| `docker-hub-proxy` | proxy | پروکسی Docker Hub |
| `docker-arvancloud-proxy` | proxy | پروکسی ArvanCloud |
| `docker-all` | group | گروه همه ریپوها |
### ریپوزیتوری‌های NuGet
| نام | نوع | توضیح |
|-----|------|-------|
| `foursat-nuget-hosted` | hosted | پکیج‌های پروتوباف |
| `nuget.org-proxy` | proxy | پروکسی NuGet.org |
| `nuget-runflare-proxy` | proxy | پروکسی Runflare |
| `nuget-group` | group | گروه همه ریپوها |
---
## 🪞 Mirror های ایرانی (Fallback)
### Docker
```
https://docker.arvancloud.ir
```
### APT/Ubuntu
```
http://mirror.arvancloud.ir/ubuntu
```
### NuGet
```
https://mirror-nuget.runflare.com/v3/index.json
```
### PyPI
```
https://mirror-pypi.runflare.com/simple
```
### NPM
```
https://mirror-npm.runflare.com
```
---
## 📦 ایمیج‌های ذخیره شده در Nexus
| ایمیج | تگ | سایز تقریبی |
|-------|-----|-------------|
| `gitea/gitea` | `1.25.3` | ~78MB |
| `mcr.microsoft.com/mssql/server` | `2022-CU16-ubuntu-22.04` | ~1.6GB |
| `gitea/act_runner` | `0.2.11`, `latest` | ~50MB |
| `registry.k8s.io/ingress-nginx/controller` | `v1.14.1` | ~280MB |
| `dotnet/sdk` | `9.0` | ~900MB |
| `dotnet/aspnet` | `9.0` | ~220MB |
| `library/nginx` | `alpine` | ~40MB |
| `docker` | `dind` | ~400MB |
| `docker-sshpass` | `latest` | ~500MB |
---
## ⚙️ تنظیمات K3s
### فایل: `/etc/rancher/k3s/registries.yaml`
```yaml
# Registry Mirrors Configuration
# Primary: Nexus (194.5.195.53:32082)
# Fallback: ArvanCloud (docker.arvancloud.ir)
mirrors:
"docker.io":
endpoint:
- "http://194.5.195.53:32082"
- "https://docker.arvancloud.ir"
- "https://registry-1.docker.io"
"194.5.195.53:32082":
endpoint:
- "http://194.5.195.53:32082"
"ghcr.io":
endpoint:
- "http://194.5.195.53:32082"
- "https://docker.arvancloud.ir"
"gcr.io":
endpoint:
- "http://194.5.195.53:32082"
- "https://docker.arvancloud.ir"
"registry.k8s.io":
endpoint:
- "http://194.5.195.53:32082"
- "https://docker.arvancloud.ir"
"quay.io":
endpoint:
- "http://194.5.195.53:32082"
- "https://docker.arvancloud.ir"
"mcr.microsoft.com":
endpoint:
- "http://194.5.195.53:32082"
- "https://docker.arvancloud.ir"
configs:
"194.5.195.53:32082":
auth:
username: admin
password: 87zH26nbqT
```
### اعمال تغییرات
```bash
sudo systemctl restart k3s
```
---
## 📝 تنظیمات APT
### فایل: `/etc/apt/sources.list.d/ubuntu.sources`
```
Types: deb
URIs: http://mirror.arvancloud.ir/ubuntu http://archive.ubuntu.com/ubuntu
Suites: noble noble-updates noble-backports
Components: main restricted universe multiverse
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg
Types: deb
URIs: http://mirror.arvancloud.ir/ubuntu http://security.ubuntu.com/ubuntu
Suites: noble-security
Components: main restricted universe multiverse
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg
```
---
## 🐍 تنظیمات PIP
### فایل: `/root/.config/pip/pip.conf`
```ini
[global]
index-url = https://pypi.org/simple
extra-index-url = https://mirror-pypi.runflare.com/simple
trusted-host = mirror-pypi.runflare.com
timeout = 60
```
---
## 📦 تنظیمات NPM
### فایل: `/root/.npmrc`
```
registry=https://registry.npmjs.org/
# Fallback (uncomment if needed):
# registry=https://mirror-npm.runflare.com
```
---
## 🔧 تنظیمات Gitea Runner
### مشکل: Runner نمیتونه از Nexus (HTTP) pull کنه
**علت:** Docker daemon داخل Runner سعی میکنه با HTTPS وصل بشه.
**راه حل:** ConfigMap برای daemon.json
### ConfigMap
```yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: docker-daemon-config
data:
daemon.json: |
{
"insecure-registries": ["194.5.195.53:32082", "194.5.195.53:30080"]
}
```
### Deployment Patch
```bash
kubectl patch deployment gitea-runner --type=json -p='[
{
"op": "add",
"path": "/spec/template/spec/volumes/-",
"value": {
"name": "docker-config",
"configMap": {
"name": "docker-daemon-config"
}
}
},
{
"op": "add",
"path": "/spec/template/spec/containers/0/volumeMounts/-",
"value": {
"name": "docker-config",
"mountPath": "/etc/docker/daemon.json",
"subPath": "daemon.json"
}
}
]'
```
### بررسی
```bash
kubectl exec $(kubectl get pods -l app=gitea-runner -o jsonpath='{.items[0].metadata.name}') \
-c docker -- docker info | grep -A 5 'Insecure Registries'
```
---
## 📁 ساختار Dockerfile ها
### الگوی استاندارد (با Nexus)
```dockerfile
FROM 194.5.195.53:32082/dotnet/sdk:9.0 AS build
WORKDIR /src
# Copy NuGet config
COPY src/NuGet.config ./
# Restore and build
RUN dotnet restore "Project.csproj" --configfile NuGet.config
RUN dotnet publish "Project.csproj" -c Release -o /app/publish --no-restore
FROM 194.5.195.53:32082/dotnet/aspnet:9.0 AS runtime
WORKDIR /app
COPY --from=build /app/publish .
ENTRYPOINT ["dotnet", "Project.dll"]
```
### NuGet.config
```xml
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<clear />
<add key="nexus" value="http://194.5.195.53:32081/repository/nuget-group/index.json" />
</packageSources>
</configuration>
```
---
## 🔄 ساختار Workflow (CI/CD)
### الگوی استاندارد `kub-deploy.yml`
```yaml
name: Build and Deploy
on:
push:
branches:
- kub-stage # یا production
env:
REGISTRY: 194.5.195.53:30080
IMAGE_NAME: admin/project-name
K8S_SERVER: 194.5.195.53 # یا 45.149.79.127 برای Production
jobs:
build-and-deploy:
runs-on: ubuntu-latest
container:
image: 194.5.195.53:32082/docker-sshpass:latest
options: --privileged
steps:
- name: Start Docker daemon
run: |
mkdir -p /etc/docker
cat > /etc/docker/daemon.json << 'DAEMON'
{
"insecure-registries": ["194.5.195.53:30080", "194.5.195.53:32082"]
}
DAEMON
dockerd &
for i in $(seq 1 90); do
docker info >/dev/null 2>&1 && break || sleep 2
done
- name: Checkout code
run: |
git clone --depth 1 --branch $BRANCH http://gitea-svc:3000/admin/PROJECT.git .
- name: Build Docker Image
run: |
docker build -t ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest .
- name: Push to Registry
run: |
echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login ${{ env.REGISTRY }} -u admin --password-stdin
docker push ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest
- name: Deploy
run: |
sshpass -p "${{ secrets.K8S_SSH_PASSWORD }}" ssh -o StrictHostKeyChecking=no root@${{ env.K8S_SERVER }} \
"kubectl rollout restart deployment/PROJECT"
```
---
## 🔐 Secrets مورد نیاز در Gitea
| Secret | مقدار | توضیح |
|--------|-------|-------|
| `REGISTRY_PASSWORD` | `87zH26nbqT` | رمز Gitea Registry |
| `K8S_SSH_PASSWORD` | `87zH26nbqT` | رمز SSH سرور |
---
## 💾 بکاپ روزانه MSSQL
### CronJob
```yaml
apiVersion: batch/v1
kind: CronJob
metadata:
name: mssql-backup
spec:
schedule: "0 2 * * *" # هر روز ساعت 2 صبح
jobTemplate:
spec:
template:
spec:
containers:
- name: backup
image: 194.5.195.53:32082/mcr.microsoft.com/mssql-tools:latest
command:
- /bin/bash
- -c
- |
DATE=$(date +%Y%m%d)
for DB in gitea Foursat; do
/opt/mssql-tools/bin/sqlcmd -S mssql-svc -U sa -P '87zH26nbqT' \
-Q "BACKUP DATABASE [$DB] TO DISK='/backups/${DB}_${DATE}.bak'"
done
volumeMounts:
- name: backup-volume
mountPath: /backups
volumes:
- name: backup-volume
hostPath:
path: /mnt/mssql-backups
restartPolicy: OnFailure
```
---
## 📊 خلاصه پروژه‌ها
| پروژه | Dockerfile | Workflow Stage | Workflow Prod |
|-------|------------|----------------|---------------|
| BackOffice | ✅ Nexus | ✅ | ✅ |
| BackOffice.BFF | ✅ Nexus | ✅ | ✅ |
| CMS | ✅ Nexus | ✅ | ✅ |
| FrontOffice | ✅ Nexus | ✅ | ✅ |
| FrontOffice.BFF | ✅ Nexus | ✅ | ✅ |
---
## 🚨 Troubleshooting
### مشکل: Image pull failed - HTTPS error
```
Error: http: server gave HTTP response to HTTPS client
```
**راه حل:** اضافه کردن registry به insecure-registries
### مشکل: NuGet restore failed
**راه حل:** بررسی NuGet.config و اتصال به Nexus
### مشکل: Runner CrashLoopBackOff
**راه حل:** بررسی لاگ‌ها با `kubectl logs`
### مشکل: K3s نمیتونه pull کنه
**راه حل:** بررسی `/etc/rancher/k3s/registries.yaml` و restart K3s
---
## 📞 دستورات مفید
### بررسی وضعیت Runner
```bash
kubectl get pods -l app=gitea-runner
kubectl logs -l app=gitea-runner -c runner --tail=50
```
### تست pull از Nexus
```bash
crictl pull 194.5.195.53:32082/dotnet/sdk:9.0
```
### بررسی ایمیج‌ها در Nexus
```bash
curl -u admin:87zH26nbqT http://194.5.195.53:32082/v2/_catalog
```
### Restart K3s
```bash
sudo systemctl restart k3s
```
---
## 📅 تاریخچه تغییرات
| تاریخ | تغییر |
|-------|-------|
| 2026-01-29 | راه‌اندازی اولیه، تنظیم Nexus، Runner، و Mirror ها |
| 2026-01-29 | تنظیم Production server برای استفاده از Stage Nexus |
| 2026-01-29 | آپدیت Dockerfile ها و Workflow های production |
| 2026-01-29 | فیکس insecure registry برای Gitea Runner |
---
> 📝 این داکیومنت توسط Copilot تهیه شده و باید با تغییرات پروژه بروزرسانی شود.
---
# تنظیمات Nexus (جزئیات کامل)
# ✅ Nexus Repository Manager - Complete Setup
## 📦 Deployed Services
### Nexus Repository Manager
- **Version:** 3.38.0 (Compatible with x86-64-v1 CPU)
- **Web UI:** https://nexus.se.kbs1.ir
- **NodePort:** http://194.5.195.53:32081
- **Credentials:** admin / 87zH26nbqT
### Kubernetes Resources
```bash
# Pod
kubectl get pod | grep nexus
# nexus-6575454f69-fv29t 1/1 Running
# Service (NodePort)
kubectl get svc nexus
# Ports: 8081:32081 (Web UI)
# 8082:32082 (Docker Hosted)
# 8083:32083 (Docker Proxy)
# 8084:32084 (Docker Group)
# Ingress
kubectl get ingress nexus-ingress
# Host: nexus.se.kbs1.ir
# TLS: Self-signed certificate (via cert-manager)
```
---
## 📦 Repositories Created
### NuGet Repositories
1. **nuget-org-proxy** (Proxy)
- Proxies: https://api.nuget.org/v3/index.json
- Caches packages from nuget.org
- URL: https://nexus.se.kbs1.ir/repository/nuget-org-proxy/index.json
2. **foursat-nuget-hosted** (Hosted)
- For private FourSat packages
- URL: https://nexus.se.kbs1.ir/repository/foursat-nuget-hosted/index.json
3. **nuget-all** (Group)
- Combines: nuget-org-proxy + foursat-nuget-hosted
- **Use this URL in projects**
- URL: https://nexus.se.kbs1.ir/repository/nuget-all/index.json
### Docker Repositories
1. **docker-hosted** (Hosted)
- For private Docker images
- Port: 32082
- URL: 194.5.195.53:32082
2. **docker-hub-proxy** (Proxy)
- Proxies: https://registry-1.docker.io (Docker Hub)
- Caches images from Docker Hub
- Port: 32083
- URL: 194.5.195.53:32083
3. **docker-all** (Group)
- Combines: docker-hosted + docker-hub-proxy
- Port: 32084
- **Use this for Kubernetes**
- URL: 194.5.195.53:32084
---
## 🔧 Project Configuration
### NuGet.config (Already Updated)
All projects now use Nexus as primary source:
```xml
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<clear />
<!-- Nexus as primary source (proxies nuget.org + caches packages) -->
<add key="Nexus" value="https://nexus.se.kbs1.ir/repository/nuget-all/index.json" />
<!-- Backup: Direct Gitea registries -->
<add key="FourSat" value="https://git.afrino.co/api/packages/FourSat/nuget/index.json" />
<add key="Afrino" value="https://git.afrino.co/api/packages/Afrino/nuget/index.json" />
</packageSources>
<packageSourceCredentials>
<Nexus>
<add key="Username" value="admin" />
<add key="ClearTextPassword" value="87zH26nbqT" />
</Nexus>
<FourSat>
<add key="Username" value="masoud" />
<add key="ClearTextPassword" value="87zH26nbqT" />
</FourSat>
<Afrino>
<add key="Username" value="systemuser" />
<add key="ClearTextPassword" value="sZSA7PTiv3pUSQZ" />
</Afrino>
</packageSourceCredentials>
</configuration>
```
**Updated files:**
-`/BackOffice/src/BackOffice/NuGet.config`
-`/BackOffice.BFF/src/BackOffice.BFF.WebApi/NuGet.config`
-`/FrontOffice/src/FrontOffice.Main/NuGet.config`
-`/FrontOffice.BFF/src/FrontOffice.BFF.WebApi/NuGet.config`
---
## 🐳 Docker Registry Configuration
### For Kubernetes Deployments
Update `/etc/containerd/config.toml` on all nodes:
```toml
[plugins."io.containerd.grpc.v1.cri".registry]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."194.5.195.53:32084"]
endpoint = ["http://194.5.195.53:32084"]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
endpoint = ["http://194.5.195.53:32084"]
[plugins."io.containerd.grpc.v1.cri".registry.configs]
[plugins."io.containerd.grpc.v1.cri".registry.configs."194.5.195.53:32084".auth]
username = "admin"
password = "87zH26nbqT"
```
Then restart containerd:
```bash
systemctl restart containerd
```
### For Docker
Add to `/etc/docker/daemon.json`:
```json
{
"insecure-registries": [
"194.5.195.53:32082",
"194.5.195.53:32083",
"194.5.195.53:32084"
],
"registry-mirrors": [
"http://194.5.195.53:32084"
]
}
```
Then restart Docker:
```bash
systemctl restart docker
```
### Docker Login
```bash
docker login 194.5.195.53:32084 -u admin -p 87zH26nbqT
docker login 194.5.195.53:32082 -u admin -p 87zH26nbqT
docker login 194.5.195.53:32083 -u admin -p 87zH26nbqT
```
---
## 🚀 Usage Examples
### Pull Docker Images via Nexus Proxy
```bash
# Instead of: docker pull nginx:alpine
docker pull 194.5.195.53:32084/nginx:alpine
# Instead of: docker pull mcr.microsoft.com/dotnet/aspnet:9.0
docker pull 194.5.195.53:32084/mcr.microsoft.com/dotnet/aspnet:9.0
```
**First pull:** Downloads from Docker Hub and caches in Nexus
**Subsequent pulls:** Served from Nexus cache (no internet needed)
### Push Private Docker Images
```bash
# Tag image
docker tag myapp:latest 194.5.195.53:32082/myapp:latest
# Push to hosted repository
docker push 194.5.195.53:32082/myapp:latest
```
### NuGet Package Restore
```bash
cd /path/to/project
dotnet restore
```
**First restore:** Downloads from nuget.org via Nexus proxy
**Subsequent restores:** Served from Nexus cache (no internet needed)
### Publish Private NuGet Packages
```bash
# Pack project
dotnet pack MyProject.csproj -c Release
# Push to Nexus hosted repository
dotnet nuget push MyProject.1.0.0.nupkg \
--source https://nexus.se.kbs1.ir/repository/foursat-nuget-hosted/ \
--api-key admin:87zH26nbqT
```
---
## 🔍 Verification
### Check NuGet Sources
```bash
dotnet nuget list source
```
Expected output:
```
Registered Sources:
1. Nexus [Enabled]
https://nexus.se.kbs1.ir/repository/nuget-all/index.json
2. FourSat [Enabled]
https://git.afrino.co/api/packages/FourSat/nuget/index.json
3. Afrino [Enabled]
https://git.afrino.co/api/packages/Afrino/nuget/index.json
```
### Test Package Download
```bash
# This should use Nexus as primary source
dotnet add package Newtonsoft.Json
# Check Nexus logs
kubectl logs nexus-6575454f69-fv29t | tail -20
```
### Check Cached Packages in Nexus
```bash
# SSH to server
ssh root@194.5.195.53
# Check blob storage
du -sh /var/lib/nexus/blobs/default/content/*
```
---
## 📊 Benefits
### NuGet Caching
- ✅ Packages download once, cached forever
- ✅ No repeated downloads from nuget.org
- ✅ Faster CI/CD builds
- ✅ Works offline after first download
### Docker Caching
- ✅ Base images cached locally (aspnet, sdk, nginx, etc.)
- ✅ No repeated downloads from Docker Hub
- ✅ Faster Kubernetes deployments
- ✅ Works offline after first pull
### Private Package Hosting
- ✅ Host private NuGet packages
- ✅ Host private Docker images
- ✅ Version control for artifacts
- ✅ Access control via credentials
---
## 🛠️ Maintenance
### Check Repository Storage
Via UI:
1. Login to https://nexus.se.kbs1.ir
2. Go to: ⚙️ Settings → System → Blob Stores
3. View: Storage usage per blob store
Via API:
```bash
curl -u admin:87zH26nbqT \
http://194.5.195.53:32081/service/rest/v1/blobstores
```
### Clear Cache (if needed)
Via UI:
1. Go to: ⚙️ Settings → Repository → Repositories
2. Select repository (e.g., `nuget-org-proxy`)
3. Click: **Delete cache**
### Backup Nexus Data
```bash
# Stop Nexus
kubectl scale deployment nexus --replicas=0
# Backup data
tar -czf nexus-backup-$(date +%Y%m%d).tar.gz /var/lib/nexus/
# Start Nexus
kubectl scale deployment nexus --replicas=1
```
---
## 📝 Files Created
-`/deployment/nexus-k8s.yaml` - Kubernetes deployment
-`/deployment/nexus-ingress.yaml` - Ingress with TLS
-`/deployment/create-nexus-repos.sh` - Repository creation script
-`/deployment/NEXUS-COMPLETE-SETUP.md` - This document
---
## 🎯 Next Steps
1. **Test NuGet Caching:**
```bash
cd BackOffice/src
dotnet clean
rm -rf ~/.nuget/packages
dotnet restore
# Check Nexus UI → Browse → nuget-org-proxy
```
2. **Configure Kubernetes to use Docker proxy:**
```bash
# Update containerd config (see Docker Registry Configuration above)
systemctl restart containerd
# Pull image via Nexus
crictl pull 194.5.195.53:32084/nginx:alpine
```
3. **Update Dockerfiles to use local images:**
```dockerfile
# Instead of: FROM mcr.microsoft.com/dotnet/aspnet:9.0
FROM 194.5.195.53:32084/mcr.microsoft.com/dotnet/aspnet:9.0
```
4. **Update CI/CD workflows:**
- Already using local registry: `194.5.195.53:32500`
- Can migrate to Nexus Docker registry: `194.5.195.53:32084`
---
## ✅ Summary
**Deployed:** Nexus Repository Manager 3.38.0
**Accessible:** https://nexus.se.kbs1.ir (with TLS)
**Repositories:** NuGet (proxy, hosted, group) + Docker (proxy, hosted, group)
**Projects Updated:** All 4 NuGet.config files now use Nexus as primary source
**Status:** Ready for production use
**Result:** Complete offline deployment capability for both NuGet packages and Docker images! 🎉