admin/docs
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
ce74377012117db00a37a8054119ed0882f14358
docs/deployment/OFFLINE-DEPLOYMENT-GUIDE.md
T

20 KiB
Raw Blame History

راهنمای دیپلوی آفلاین FourSat

تاریخ: 2026-01-29 هدف: دیپلوی بدون نیاز به اینترنت خارجی

📋 خلاصه اجرایی

این راهنما شامل تنظیمات لازم برای دیپلوی کامل آفلاین پروژه FourSat است. با استفاده از Nexus به عنوان registry مرکزی و mirror های ایرانی به عنوان fallback، نیازی به اینترنت خارجی نیست.

🖥️ سرورها

سرور IP نقش رمز عبور
Stage 194.5.195.53 Nexus, Gitea, Runner 87zH26nbqT
Production 45.149.79.127 K8S Production 87zH26nbqT

🐳 Nexus Registry

پورت‌ها

سرویس پورت پروتکل
Nexus UI 32081 HTTP
Docker Registry 32082 HTTP (insecure)
NuGet 32081/repository/nuget-group/index.json HTTP

Credentials

Username: admin
Password: 87zH26nbqT

ریپوزیتوری‌های Docker

نام نوع توضیح
docker-hosted hosted ایمیج‌های پروژه
docker-hub-proxy proxy پروکسی Docker Hub
docker-arvancloud-proxy proxy پروکسی ArvanCloud
docker-all group گروه همه ریپوها

ریپوزیتوری‌های NuGet

نام نوع توضیح
foursat-nuget-hosted hosted پکیج‌های پروتوباف
nuget.org-proxy proxy پروکسی NuGet.org
nuget-runflare-proxy proxy پروکسی Runflare
nuget-group group گروه همه ریپوها

🪞 Mirror های ایرانی (Fallback)

Docker

https://docker.arvancloud.ir

APT/Ubuntu

http://mirror.arvancloud.ir/ubuntu

NuGet

https://mirror-nuget.runflare.com/v3/index.json

PyPI

https://mirror-pypi.runflare.com/simple

NPM

https://mirror-npm.runflare.com

📦 ایمیج‌های ذخیره شده در Nexus

ایمیج تگ سایز تقریبی
gitea/gitea 1.25.3 ~78MB
mcr.microsoft.com/mssql/server 2022-CU16-ubuntu-22.04 ~1.6GB
gitea/act_runner 0.2.11, latest ~50MB
registry.k8s.io/ingress-nginx/controller v1.14.1 ~280MB
dotnet/sdk 9.0 ~900MB
dotnet/aspnet 9.0 ~220MB
library/nginx alpine ~40MB
docker dind ~400MB
docker-sshpass latest ~500MB

⚙️ تنظیمات K3s

فایل: /etc/rancher/k3s/registries.yaml

# Registry Mirrors Configuration
# Primary: Nexus (194.5.195.53:32082)
# Fallback: ArvanCloud (docker.arvancloud.ir)

mirrors:
  "docker.io":
    endpoint:
      - "http://194.5.195.53:32082"
      - "https://docker.arvancloud.ir"
      - "https://registry-1.docker.io"
  "194.5.195.53:32082":
    endpoint:
      - "http://194.5.195.53:32082"
  "ghcr.io":
    endpoint:
      - "http://194.5.195.53:32082"
      - "https://docker.arvancloud.ir"
  "gcr.io":
    endpoint:
      - "http://194.5.195.53:32082"
      - "https://docker.arvancloud.ir"
  "registry.k8s.io":
    endpoint:
      - "http://194.5.195.53:32082"
      - "https://docker.arvancloud.ir"
  "quay.io":
    endpoint:
      - "http://194.5.195.53:32082"
      - "https://docker.arvancloud.ir"
  "mcr.microsoft.com":
    endpoint:
      - "http://194.5.195.53:32082"
      - "https://docker.arvancloud.ir"

configs:
  "194.5.195.53:32082":
    auth:
      username: admin
      password: 87zH26nbqT

اعمال تغییرات

sudo systemctl restart k3s

📝 تنظیمات APT

فایل: /etc/apt/sources.list.d/ubuntu.sources

Types: deb
URIs: http://mirror.arvancloud.ir/ubuntu http://archive.ubuntu.com/ubuntu
Suites: noble noble-updates noble-backports
Components: main restricted universe multiverse
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg

Types: deb
URIs: http://mirror.arvancloud.ir/ubuntu http://security.ubuntu.com/ubuntu
Suites: noble-security
Components: main restricted universe multiverse
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg

🐍 تنظیمات PIP

فایل: /root/.config/pip/pip.conf

[global]
index-url = https://pypi.org/simple
extra-index-url = https://mirror-pypi.runflare.com/simple
trusted-host = mirror-pypi.runflare.com
timeout = 60

📦 تنظیمات NPM

فایل: /root/.npmrc

registry=https://registry.npmjs.org/
# Fallback (uncomment if needed):
# registry=https://mirror-npm.runflare.com

🔧 تنظیمات Gitea Runner

مشکل: Runner نمیتونه از Nexus (HTTP) pull کنه

علت: Docker daemon داخل Runner سعی میکنه با HTTPS وصل بشه.

راه حل: ConfigMap برای daemon.json

ConfigMap

apiVersion: v1
kind: ConfigMap
metadata:
  name: docker-daemon-config
data:
  daemon.json: |
    {
      "insecure-registries": ["194.5.195.53:32082", "194.5.195.53:30080"]
    }

Deployment Patch

kubectl patch deployment gitea-runner --type=json -p='[
  {
    "op": "add",
    "path": "/spec/template/spec/volumes/-",
    "value": {
      "name": "docker-config",
      "configMap": {
        "name": "docker-daemon-config"
      }
    }
  },
  {
    "op": "add",
    "path": "/spec/template/spec/containers/0/volumeMounts/-",
    "value": {
      "name": "docker-config",
      "mountPath": "/etc/docker/daemon.json",
      "subPath": "daemon.json"
    }
  }
]'

بررسی

kubectl exec $(kubectl get pods -l app=gitea-runner -o jsonpath='{.items[0].metadata.name}') \
  -c docker -- docker info | grep -A 5 'Insecure Registries'

📁 ساختار Dockerfile ها

الگوی استاندارد (با Nexus)

FROM 194.5.195.53:32082/dotnet/sdk:9.0 AS build
WORKDIR /src

# Copy NuGet config
COPY src/NuGet.config ./

# Restore and build
RUN dotnet restore "Project.csproj" --configfile NuGet.config
RUN dotnet publish "Project.csproj" -c Release -o /app/publish --no-restore

FROM 194.5.195.53:32082/dotnet/aspnet:9.0 AS runtime
WORKDIR /app
COPY --from=build /app/publish .
ENTRYPOINT ["dotnet", "Project.dll"]

NuGet.config

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <clear />
    <add key="nexus" value="http://194.5.195.53:32081/repository/nuget-group/index.json" />
  </packageSources>
</configuration>

🔄 ساختار Workflow (CI/CD)

الگوی استاندارد kub-deploy.yml

name: Build and Deploy

on:
  push:
    branches:
      - kub-stage  # یا production

env:
  REGISTRY: 194.5.195.53:30080
  IMAGE_NAME: admin/project-name
  K8S_SERVER: 194.5.195.53  # یا 45.149.79.127 برای Production

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    container:
      image: 194.5.195.53:32082/docker-sshpass:latest
      options: --privileged
    steps:
      - name: Start Docker daemon
        run: |
          mkdir -p /etc/docker
          cat > /etc/docker/daemon.json << 'DAEMON'
          {
            "insecure-registries": ["194.5.195.53:30080", "194.5.195.53:32082"]
          }
          DAEMON
          dockerd &
          for i in $(seq 1 90); do 
            docker info >/dev/null 2>&1 && break || sleep 2
          done

      - name: Checkout code
        run: |
          git clone --depth 1 --branch $BRANCH http://gitea-svc:3000/admin/PROJECT.git .

      - name: Build Docker Image
        run: |
          docker build -t ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest .

      - name: Push to Registry
        run: |
          echo "${{ secrets.REGISTRY_PASSWORD }}" | docker login ${{ env.REGISTRY }} -u admin --password-stdin
          docker push ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:latest

      - name: Deploy
        run: |
          sshpass -p "${{ secrets.K8S_SSH_PASSWORD }}" ssh -o StrictHostKeyChecking=no root@${{ env.K8S_SERVER }} \
            "kubectl rollout restart deployment/PROJECT"

🔐 Secrets مورد نیاز در Gitea

Secret مقدار توضیح
REGISTRY_PASSWORD 87zH26nbqT رمز Gitea Registry
K8S_SSH_PASSWORD 87zH26nbqT رمز SSH سرور

💾 بکاپ روزانه MSSQL

CronJob

apiVersion: batch/v1
kind: CronJob
metadata:
  name: mssql-backup
spec:
  schedule: "0 2 * * *"  # هر روز ساعت 2 صبح
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: backup
            image: 194.5.195.53:32082/mcr.microsoft.com/mssql-tools:latest
            command:
            - /bin/bash
            - -c
            - |
              DATE=$(date +%Y%m%d)
              for DB in gitea Foursat; do
                /opt/mssql-tools/bin/sqlcmd -S mssql-svc -U sa -P '87zH26nbqT' \
                  -Q "BACKUP DATABASE [$DB] TO DISK='/backups/${DB}_${DATE}.bak'"
              done
            volumeMounts:
            - name: backup-volume
              mountPath: /backups
          volumes:
          - name: backup-volume
            hostPath:
              path: /mnt/mssql-backups
          restartPolicy: OnFailure

📊 خلاصه پروژه‌ها

پروژه Dockerfile Workflow Stage Workflow Prod
BackOffice ✅ Nexus ✅ ✅
BackOffice.BFF ✅ Nexus ✅ ✅
CMS ✅ Nexus ✅ ✅
FrontOffice ✅ Nexus ✅ ✅
FrontOffice.BFF ✅ Nexus ✅ ✅

🚨 Troubleshooting

مشکل: Image pull failed - HTTPS error

Error: http: server gave HTTP response to HTTPS client

راه حل: اضافه کردن registry به insecure-registries

مشکل: NuGet restore failed

راه حل: بررسی NuGet.config و اتصال به Nexus

مشکل: Runner CrashLoopBackOff

راه حل: بررسی لاگ‌ها با kubectl logs

مشکل: K3s نمیتونه pull کنه

راه حل: بررسی /etc/rancher/k3s/registries.yaml و restart K3s

📞 دستورات مفید

بررسی وضعیت Runner

kubectl get pods -l app=gitea-runner
kubectl logs -l app=gitea-runner -c runner --tail=50

تست pull از Nexus

crictl pull 194.5.195.53:32082/dotnet/sdk:9.0

بررسی ایمیج‌ها در Nexus

curl -u admin:87zH26nbqT http://194.5.195.53:32082/v2/_catalog

Restart K3s

sudo systemctl restart k3s

📅 تاریخچه تغییرات

تاریخ تغییر
2026-01-29 راه‌اندازی اولیه، تنظیم Nexus، Runner، و Mirror ها
2026-01-29 تنظیم Production server برای استفاده از Stage Nexus
2026-01-29 آپدیت Dockerfile ها و Workflow های production
2026-01-29 فیکس insecure registry برای Gitea Runner

📝 این داکیومنت توسط Copilot تهیه شده و باید با تغییرات پروژه بروزرسانی شود.

تنظیمات Nexus (جزئیات کامل)

✅ Nexus Repository Manager - Complete Setup

📦 Deployed Services

Nexus Repository Manager

Kubernetes Resources

# Pod
kubectl get pod | grep nexus
# nexus-6575454f69-fv29t   1/1   Running

# Service (NodePort)
kubectl get svc nexus
# Ports: 8081:32081 (Web UI)
#        8082:32082 (Docker Hosted)
#        8083:32083 (Docker Proxy)
#        8084:32084 (Docker Group)

# Ingress
kubectl get ingress nexus-ingress
# Host: nexus.se.kbs1.ir
# TLS: Self-signed certificate (via cert-manager)

📦 Repositories Created

NuGet Repositories

  1. nuget-org-proxy (Proxy)

  2. foursat-nuget-hosted (Hosted)

  3. nuget-all (Group)

Docker Repositories

  1. docker-hosted (Hosted)

    • For private Docker images
    • Port: 32082
    • URL: 194.5.195.53:32082

  2. docker-hub-proxy (Proxy)

  3. docker-all (Group)

    • Combines: docker-hosted + docker-hub-proxy
    • Port: 32084
    • Use this for Kubernetes
    • URL: 194.5.195.53:32084

🔧 Project Configuration

NuGet.config (Already Updated)

All projects now use Nexus as primary source:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <clear />
    <!-- Nexus as primary source (proxies nuget.org + caches packages) -->
    <add key="Nexus" value="https://nexus.se.kbs1.ir/repository/nuget-all/index.json" />
    <!-- Backup: Direct Gitea registries -->
    <add key="FourSat" value="https://git.afrino.co/api/packages/FourSat/nuget/index.json" />
    <add key="Afrino" value="https://git.afrino.co/api/packages/Afrino/nuget/index.json" />
  </packageSources>
  <packageSourceCredentials>
    <Nexus>
      <add key="Username" value="admin" />
      <add key="ClearTextPassword" value="87zH26nbqT" />
    </Nexus>
    <FourSat>
      <add key="Username" value="masoud" />
      <add key="ClearTextPassword" value="87zH26nbqT" />
    </FourSat>
    <Afrino>
      <add key="Username" value="systemuser" />
      <add key="ClearTextPassword" value="sZSA7PTiv3pUSQZ" />
    </Afrino>
  </packageSourceCredentials>
</configuration>

Updated files:

  • ✅ /BackOffice/src/BackOffice/NuGet.config
  • ✅ /BackOffice.BFF/src/BackOffice.BFF.WebApi/NuGet.config
  • ✅ /FrontOffice/src/FrontOffice.Main/NuGet.config
  • ✅ /FrontOffice.BFF/src/FrontOffice.BFF.WebApi/NuGet.config

🐳 Docker Registry Configuration

For Kubernetes Deployments

Update /etc/containerd/config.toml on all nodes:

[plugins."io.containerd.grpc.v1.cri".registry]
  [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
    [plugins."io.containerd.grpc.v1.cri".registry.mirrors."194.5.195.53:32084"]
      endpoint = ["http://194.5.195.53:32084"]
    [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
      endpoint = ["http://194.5.195.53:32084"]
  
  [plugins."io.containerd.grpc.v1.cri".registry.configs]
    [plugins."io.containerd.grpc.v1.cri".registry.configs."194.5.195.53:32084".auth]
      username = "admin"
      password = "87zH26nbqT"

Then restart containerd:

systemctl restart containerd

For Docker

Add to /etc/docker/daemon.json:

{
  "insecure-registries": [
    "194.5.195.53:32082",
    "194.5.195.53:32083",
    "194.5.195.53:32084"
  ],
  "registry-mirrors": [
    "http://194.5.195.53:32084"
  ]
}

Then restart Docker:

systemctl restart docker

Docker Login

docker login 194.5.195.53:32084 -u admin -p 87zH26nbqT
docker login 194.5.195.53:32082 -u admin -p 87zH26nbqT
docker login 194.5.195.53:32083 -u admin -p 87zH26nbqT

🚀 Usage Examples

Pull Docker Images via Nexus Proxy

# Instead of: docker pull nginx:alpine
docker pull 194.5.195.53:32084/nginx:alpine

# Instead of: docker pull mcr.microsoft.com/dotnet/aspnet:9.0
docker pull 194.5.195.53:32084/mcr.microsoft.com/dotnet/aspnet:9.0

First pull: Downloads from Docker Hub and caches in Nexus
Subsequent pulls: Served from Nexus cache (no internet needed)

Push Private Docker Images

# Tag image
docker tag myapp:latest 194.5.195.53:32082/myapp:latest

# Push to hosted repository
docker push 194.5.195.53:32082/myapp:latest

NuGet Package Restore

cd /path/to/project
dotnet restore

First restore: Downloads from nuget.org via Nexus proxy
Subsequent restores: Served from Nexus cache (no internet needed)

Publish Private NuGet Packages

# Pack project
dotnet pack MyProject.csproj -c Release

# Push to Nexus hosted repository
dotnet nuget push MyProject.1.0.0.nupkg \
  --source https://nexus.se.kbs1.ir/repository/foursat-nuget-hosted/ \
  --api-key admin:87zH26nbqT

🔍 Verification

Check NuGet Sources

dotnet nuget list source

Expected output:

Registered Sources:
  1.  Nexus [Enabled]
      https://nexus.se.kbs1.ir/repository/nuget-all/index.json
  2.  FourSat [Enabled]
      https://git.afrino.co/api/packages/FourSat/nuget/index.json
  3.  Afrino [Enabled]
      https://git.afrino.co/api/packages/Afrino/nuget/index.json

Test Package Download

# This should use Nexus as primary source
dotnet add package Newtonsoft.Json

# Check Nexus logs
kubectl logs nexus-6575454f69-fv29t | tail -20

Check Cached Packages in Nexus

# SSH to server
ssh root@194.5.195.53

# Check blob storage
du -sh /var/lib/nexus/blobs/default/content/*

📊 Benefits

NuGet Caching

  • ✅ Packages download once, cached forever
  • ✅ No repeated downloads from nuget.org
  • ✅ Faster CI/CD builds
  • ✅ Works offline after first download

Docker Caching

  • ✅ Base images cached locally (aspnet, sdk, nginx, etc.)
  • ✅ No repeated downloads from Docker Hub
  • ✅ Faster Kubernetes deployments
  • ✅ Works offline after first pull

Private Package Hosting

  • ✅ Host private NuGet packages
  • ✅ Host private Docker images
  • ✅ Version control for artifacts
  • ✅ Access control via credentials

🛠️ Maintenance

Check Repository Storage

Via UI:

  1. Login to https://nexus.se.kbs1.ir
  2. Go to: ⚙️ Settings → System → Blob Stores
  3. View: Storage usage per blob store

Via API:

curl -u admin:87zH26nbqT \
  http://194.5.195.53:32081/service/rest/v1/blobstores

Clear Cache (if needed)

Via UI:

  1. Go to: ⚙️ Settings → Repository → Repositories
  2. Select repository (e.g., nuget-org-proxy)
  3. Click: Delete cache

Backup Nexus Data

# Stop Nexus
kubectl scale deployment nexus --replicas=0

# Backup data
tar -czf nexus-backup-$(date +%Y%m%d).tar.gz /var/lib/nexus/

# Start Nexus
kubectl scale deployment nexus --replicas=1

📝 Files Created

  • ✅ /deployment/nexus-k8s.yaml - Kubernetes deployment
  • ✅ /deployment/nexus-ingress.yaml - Ingress with TLS
  • ✅ /deployment/create-nexus-repos.sh - Repository creation script
  • ✅ /deployment/NEXUS-COMPLETE-SETUP.md - This document

🎯 Next Steps

  1. Test NuGet Caching:

    cd BackOffice/src
dotnet clean
rm -rf ~/.nuget/packages
dotnet restore
# Check Nexus UI → Browse → nuget-org-proxy

  2. Configure Kubernetes to use Docker proxy:

    # Update containerd config (see Docker Registry Configuration above)
systemctl restart containerd

# Pull image via Nexus
crictl pull 194.5.195.53:32084/nginx:alpine

  3. Update Dockerfiles to use local images:

    # Instead of: FROM mcr.microsoft.com/dotnet/aspnet:9.0
FROM 194.5.195.53:32084/mcr.microsoft.com/dotnet/aspnet:9.0

  4. Update CI/CD workflows:

    • Already using local registry: 194.5.195.53:32500
    • Can migrate to Nexus Docker registry: 194.5.195.53:32084

✅ Summary

Deployed: Nexus Repository Manager 3.38.0
Accessible: https://nexus.se.kbs1.ir (with TLS)
Repositories: NuGet (proxy, hosted, group) + Docker (proxy, hosted, group)
Projects Updated: All 4 NuGet.config files now use Nexus as primary source
Status: Ready for production use

Result: Complete offline deployment capability for both NuGet packages and Docker images! 🎉

Reference in New Issue View Git Blame Copy Permalink