Platform Engineering và Internal Developer Platform

🎯 Mục tiêu: Thiết kế blueprint IDP cho một tổ chức 10 team/50 developer: xác định capabilities, toolchain, và viết Architecture Decision Record (ADR) cho 2 quyết định quan trọng.

🧰 Công cụ / nền tảng: VS Code, Git, Mermaid (VS Code extension).

📦 Chuẩn bị: Git repo mới; VS Code với extension Markdown Preview Mermaid Support.

▶️ Bước 1 — Tạo cấu trúc IDP blueprint repo:

mkdir idp-blueprint && cd idp-blueprint
git init

# Cấu trúc thư mục IDP
mkdir -p docs/adr platform/templates platform/modules portal

cat > README.md <<'EOF'
# Internal Developer Platform Blueprint
## Scope
- 10 product teams, 50 developers
- Kubernetes-based on AWS EKS
- GitOps workflow (Argo CD)

## Capabilities
| Capability         | Tool              | Status   |
|--------------------|-------------------|----------|
| Self-service infra | Crossplane        | Planned  |
| CI/CD pipelines    | GitHub Actions    | Active   |
| Service catalog    | Backstage         | Planned  |
| Observability      | Prometheus/Grafana| Active   |
| Secret management  | Vault             | Active   |
| Golden path        | Cookiecutter      | Planned  |
EOF

git add README.md && git commit -m "feat: init IDP blueprint repo"

▶️ Bước 2 — Vẽ kiến trúc IDP bằng Mermaid:

cat > docs/architecture.md <<'EOF'
# IDP Architecture

```mermaid
graph TB
    subgraph Developer["Developer (Self-Service)"]
        DEV[Developer CLI / Portal]
    end

    subgraph IDP["Internal Developer Platform"]
        PORTAL[Backstage Portal]
        CATALOG[Service Catalog]
        SCAFFOLD[Golden Path Templates]
        PORTAL --> CATALOG
        PORTAL --> SCAFFOLD
    end

    subgraph Platform["Platform Layer"]
        CP[Crossplane - Infra API]
        GITOPS[Argo CD - GitOps]
        CI[GitHub Actions - CI/CD]
        OBS[Prometheus + Grafana]
        VAULT[HashiCorp Vault]
    end

    subgraph Cloud["Cloud (AWS)"]
        EKS[EKS Cluster]
        RDS[(RDS)]
        S3[(S3)]
    end

    DEV --> PORTAL
    DEV --> CI
    SCAFFOLD --> GITOPS
    CP --> EKS
    CP --> RDS
    CP --> S3
    GITOPS --> EKS
    OBS --> EKS
```
EOF

git add docs/architecture.md && git commit -m "docs: add IDP architecture diagram"

▶️ Bước 3 — Viết ADR (Architecture Decision Record):

cat > docs/adr/ADR-001-crossplane-vs-terraform.md <<'EOF'
# ADR-001: Crossplane vs Terraform cho Self-Service Infra

## Status: Accepted
## Date: 2026-05-23

## Context
Platform Team cần cơ chế self-service để developer tạo DB, bucket
mà không cần ticket. Hai ứng viên: Crossplane (Kubernetes-native) và
Terraform Cloud với module catalog.

## Decision
Chọn Crossplane XRD làm self-service API vì:
- Native Kubernetes — developer dùng kubectl/YAML quen thuộc
- GitOps-friendly: Argo CD detect và reconcile resource tự động
- Composite Resource che đi AWS API complexity khỏi developer

## Consequences
- Platform Team cần học Crossplane XRD (1–2 sprint ramp-up)
- Terraform vẫn dùng cho bootstrapping cluster ban đầu
- Developer không cần biết AWS API trực tiếp — giảm cognitive load
EOF

git add docs/adr/ && git commit -m "docs: add ADR-001 crossplane vs terraform decision"

✅ Kết quả mong đợi: Repo có cấu trúc blueprint rõ ràng. git log --oneline hiển thị 3 commits. Mermaid diagram render đúng trong VS Code Preview. ADR-001 ghi rõ context/decision/consequences.

🧹 Cleanup: git add . && git commit -m "lab-146: complete IDP blueprint" rồi push lên GitHub làm portfolio.

🎯 Mục tiêu: Tạo file catalog-info.yaml chuẩn Backstage cho 3 service khác nhau (Web API, thư viện shared, hệ thống nền). Hiểu Entity kinds: Component, API, System, Group, Resource.

🧰 Công cụ / nền tảng: VS Code, Git, Backstage schema (tham chiếu docs.backstage.io).

📦 Chuẩn bị: Repo IDP blueprint từ LAB-146 (hoặc repo mới); hiểu cấu trúc YAML.

▶️ Bước 1 — catalog-info.yaml cho Web API service:

# services/payment-api/catalog-info.yaml
cat > platform/catalog/payment-api.yaml <<'EOF'
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: payment-api
  title: Payment API
  description: Core payment processing service — handles charge, refund, webhook
  annotations:
    github.com/project-slug: myorg/payment-api
    backstage.io/techdocs-ref: dir:.
    prometheus.io/alert: "payment-api-sla"
  tags:
    - payments
    - api
    - tier-1
  links:
    - url: https://grafana.internal/d/payment-api
      title: Grafana Dashboard
      icon: dashboard
    - url: https://runbook.internal/payment-api
      title: Runbook
      icon: help
spec:
  type: service
  lifecycle: production
  owner: team-payments
  system: billing-system
  dependsOn:
    - component:postgres-payments
    - component:notification-service
  providesApis:
    - payment-api-v2
EOF

▶️ Bước 2 — catalog-info.yaml cho API definition:

cat > platform/catalog/payment-api-definition.yaml <<'EOF'
apiVersion: backstage.io/v1alpha1
kind: API
metadata:
  name: payment-api-v2
  description: Payment API REST contract v2
  tags:
    - rest
    - payments
spec:
  type: openapi
  lifecycle: production
  owner: team-payments
  definition: |
    openapi: "3.0.0"
    info:
      title: Payment API
      version: "2.0.0"
    paths:
      /charge:
        post:
          summary: Process a payment charge
          responses:
            "200":
              description: Charge successful
      /refund/{id}:
        post:
          summary: Refund a transaction
          responses:
            "200":
              description: Refund initiated
EOF

▶️ Bước 3 — catalog-info.yaml cho System và Group:

cat > platform/catalog/billing-system.yaml <<'EOF'
apiVersion: backstage.io/v1alpha1
kind: System
metadata:
  name: billing-system
  description: End-to-end billing and payment infrastructure
  tags:
    - core
    - payments
spec:
  owner: team-payments
---
apiVersion: backstage.io/v1alpha1
kind: Group
metadata:
  name: team-payments
  description: Payments Engineering Team
spec:
  type: team
  profile:
    displayName: Payments Team
    email: [email protected]
  children: []
  members:
    - user:nguyen.van.a
    - user:tran.thi.b
EOF

# Validate YAML syntax
python3 -c "import yaml; [yaml.safe_load(open('platform/catalog/'+f)) for f in ['payment-api.yaml','billing-system.yaml']]" && echo "YAML valid"

git add platform/catalog/
git commit -m "feat: add service catalog metadata for payment-api, billing-system"

✅ Kết quả mong đợi: 3 YAML files pass Python YAML parse (không lỗi syntax). Mỗi Component có đầy đủ: owner, lifecycle, system, dependsOn. API kind có definition OpenAPI inline. Khi import vào Backstage thật, hiển thị dependency graph đúng.

🧹 Cleanup: git push origin main — giữ lại làm mẫu catalog cho LAB-148.

🎯 Mục tiêu: Tạo golden path template cho web API service bằng Cookiecutter — scaffold một repo mới với Dockerfile, GitHub Actions CI, Helm chart, catalog-info.yaml và observability config tích hợp sẵn.

🧰 Công cụ / nền tảng: Python (Cookiecutter), VS Code, Git, Docker.

📦 Chuẩn bị: Python 3.10+; pip install cookiecutter; Docker Desktop chạy.

▶️ Bước 1 — Tạo Cookiecutter template:

pip install cookiecutter

# Tạo cấu trúc template
mkdir golden-path-web-api && cd golden-path-web-api

# cookiecutter.json — biến template
cat > cookiecutter.json <<'EOF'
{
  "service_name": "my-service",
  "team_name": "my-team",
  "port": "8080",
  "language": ["python", "nodejs", "go"],
  "tier": ["tier-1", "tier-2", "tier-3"]
}
EOF

# Tạo cấu trúc thư mục template
mkdir -p "{{cookiecutter.service_name}}/.github/workflows"
mkdir -p "{{cookiecutter.service_name}}/helm/templates"
mkdir -p "{{cookiecutter.service_name}}/src"

▶️ Bước 2 — Tạo files template:

# Dockerfile template
cat > "{{cookiecutter.service_name}}/Dockerfile" <<'TMPL'
FROM python:3.12-slim AS base
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY src/ .
EXPOSE {{cookiecutter.port}}
HEALTHCHECK --interval=30s --timeout=5s \
  CMD curl -f http://localhost:{{cookiecutter.port}}/healthz || exit 1
CMD ["python", "main.py"]
TMPL

# catalog-info.yaml template
cat > "{{cookiecutter.service_name}}/catalog-info.yaml" <<'TMPL'
apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: {{cookiecutter.service_name}}
  description: "Auto-generated from golden path template"
  tags:
    - {{cookiecutter.tier}}
    - {{cookiecutter.language}}
spec:
  type: service
  lifecycle: development
  owner: {{cookiecutter.team_name}}
TMPL

# GitHub Actions CI template
cat > "{{cookiecutter.service_name}}/.github/workflows/ci.yml" <<'TMPL'
name: CI — {{cookiecutter.service_name}}
on:
  push:
    branches: [main, 'feature/**']
  pull_request:
    branches: [main]
jobs:
  build-test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build Docker image
        run: docker build -t {{cookiecutter.service_name}}:${{ github.sha }} .
      - name: Run tests
        run: docker run --rm {{cookiecutter.service_name}}:${{ github.sha }} python -m pytest
      - name: Scan image (Trivy)
        uses: aquasecurity/trivy-action@master
        with:
          image-ref: {{cookiecutter.service_name}}:${{ github.sha }}
          severity: CRITICAL,HIGH
TMPL

git add . && git commit -m "feat: golden path web-api template"

▶️ Bước 3 — Scaffold service mới từ template:

cd ..

# Scaffold service mới — trả lời prompt
cookiecutter ./golden-path-web-api/ \
  --no-input \
  service_name="order-service" \
  team_name="team-commerce" \
  port="3000" \
  language="nodejs" \
  tier="tier-1"

# Kiểm tra kết quả
ls -la order-service/
cat order-service/catalog-info.yaml
cat order-service/Dockerfile
cat order-service/.github/workflows/ci.yml

# Output mong đợi:
# order-service/
# ├── .github/workflows/ci.yml
# ├── Dockerfile
# ├── catalog-info.yaml
# ├── helm/
# └── src/

✅ Kết quả mong đợi: Cookiecutter tạo thư mục order-service/ với tất cả files. catalog-info.yaml chứa name: order-service, owner: team-commerce. Dockerfile có EXPOSE 3000 và HEALTHCHECK. CI workflow tham chiếu đúng service name. Developer mới có thể onboard service trong <5 phút.

🧹 Cleanup: rm -rf order-service/ nếu chỉ test; giữ golden-path-web-api/ làm template thực.

🎯 Mục tiêu: Cài Crossplane trên cluster, tạo XRD cho PostgreSQLInstance, và test developer tự provision database bằng 1 YAML file.

🧰 Công cụ / nền tảng: kubectl, Helm ≥3.10, Kubernetes cluster (k3d hoặc kind), VS Code, Git.

📦 Chuẩn bị: kubectl context hoạt động; Helm installed; cluster ≥2 CPU, 4GB RAM. (Lab này dùng provider-noop để tránh tốn chi phí cloud thật.)

▶️ Bước 1 — Cài Crossplane:

# Tạo cluster local
k3d cluster create platform-lab --agents 2
kubectl cluster-info

# Cài Crossplane bằng Helm
helm repo add crossplane-stable https://charts.crossplane.io/stable
helm repo update

helm install crossplane crossplane-stable/crossplane \
  --namespace crossplane-system \
  --create-namespace \
  --wait

# Kiểm tra
kubectl get pods -n crossplane-system
# Expected: crossplane và crossplane-rbac-manager đều Running

# Cài Crossplane CLI (kubectl extension)
curl -sL "https://raw.githubusercontent.com/crossplane/crossplane/master/install.sh" | sh
mv crossplane ~/.local/bin/
crossplane version

▶️ Bước 2 — Định nghĩa CompositeResourceDefinition (XRD):

cat > xrd-postgresql.yaml <<'EOF'
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: xpostgresqlinstances.platform.company.io
spec:
  group: platform.company.io
  names:
    kind: XPostgreSQLInstance
    plural: xpostgresqlinstances
  # Claim — developer dùng kind này trong namespace riêng
  claimNames:
    kind: PostgreSQLInstance
    plural: postgresqlinstances
  versions:
    - name: v1alpha1
      served: true
      referenceable: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                parameters:
                  type: object
                  required: [storageGB, tier]
                  properties:
                    storageGB:
                      type: integer
                      minimum: 5
                      maximum: 500
                      description: Storage in GB
                    tier:
                      type: string
                      enum: [dev, staging, prod]
                      description: Environment tier
EOF

kubectl apply -f xrd-postgresql.yaml
kubectl get xrd
# Expected: xpostgresqlinstances.platform.company.io   True   True

▶️ Bước 3 — Tạo Composition (logic provision):

cat > composition-postgresql.yaml <<'EOF'
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: postgresql-standard
  labels:
    crossplane.io/xrd: xpostgresqlinstances.platform.company.io
spec:
  compositeTypeRef:
    apiVersion: platform.company.io/v1alpha1
    kind: XPostgreSQLInstance
  resources:
    # Trong lab này dùng Object resource giả (ConfigMap)
    # Trong production: dùng provider-aws RDSInstance hoặc provider-azure SQLServer
    - name: db-config
      base:
        apiVersion: v1
        kind: ConfigMap
        metadata:
          name: db-config-placeholder
          namespace: crossplane-system
        data:
          status: "provisioning"
      patches:
        - type: FromCompositeFieldPath
          fromFieldPath: spec.parameters.tier
          toFieldPath: data.tier
        - type: FromCompositeFieldPath
          fromFieldPath: spec.parameters.storageGB
          toFieldPath: data.storageGB
          transforms:
            - type: convert
              convert:
                toType: string
EOF

kubectl apply -f composition-postgresql.yaml
kubectl get composition

▶️ Bước 4 — Developer tự tạo DB (Claim):

# Tạo namespace cho team developer
kubectl create namespace team-commerce

# Developer tạo database bằng 1 YAML — không cần ticket Ops!
cat > my-database.yaml <<'EOF'
apiVersion: platform.company.io/v1alpha1
kind: PostgreSQLInstance
metadata:
  name: orders-db
  namespace: team-commerce
spec:
  parameters:
    storageGB: 20
    tier: staging
  compositionSelector:
    matchLabels:
      crossplane.io/xrd: xpostgresqlinstances.platform.company.io
  writeConnectionSecretToRef:
    name: orders-db-conn
EOF

kubectl apply -f my-database.yaml

# Theo dõi trạng thái
kubectl get postgresqlinstance orders-db -n team-commerce -w
# Expected: orders-db   True (sau vài giây)

# Kiểm tra XR được tạo
kubectl get xpostgresqlinstance -A
kubectl describe postgresqlinstance orders-db -n team-commerce

✅ Kết quả mong đợi: XRD được tạo và ESTABLISHED=True. Composition apply thành công. Developer tạo PostgreSQLInstance claim trong namespace riêng mà không biết bất kỳ AWS API nào — Platform API đã che hết. kubectl get postgresqlinstance -n team-commerce hiển thị orders-db với READY=True.

🧹 Cleanup:

kubectl delete -f my-database.yaml
kubectl delete -f composition-postgresql.yaml
kubectl delete -f xrd-postgresql.yaml
helm uninstall crossplane -n crossplane-system
k3d cluster delete platform-lab

🎯 Mục tiêu: Thiết kế Developer Experience survey theo SPACE framework, tính scorecard service quality từ metadata, và lập improvement plan dựa trên kết quả.

🧰 Công cụ / nền tảng: VS Code, Git, PowerShell, Python (tính điểm scorecard).

📦 Chuẩn bị: Python 3.10+; repo có catalog-info.yaml từ LAB-147.

▶️ Bước 1 — Tạo DX Survey template (SPACE):

cat > dx-survey-template.md <<'EOF'
# Developer Experience Survey — SPACE Framework
*Thực hiện hàng quý, ẩn danh, 5 phút*

## S — Satisfaction
1. Bạn hài lòng với tooling hiện tại thế nào? (1–5)
2. Platform/IDP giúp bạn tiết kiệm bao nhiêu giờ/tuần? (0/1/2/4/8h)
3. Điểm NPS nội bộ: bạn có giới thiệu Platform Team cho đồng nghiệp? (0–10)

## P — Performance
4. Thời gian từ commit đến deploy production trung bình? (<1h / 1-4h / >4h)
5. Tần suất deploy production? (nhiều lần/ngày / hàng ngày / hàng tuần)

## A — Activity
6. Số PR bạn review/tuần trung bình? (1-5 / 6-10 / >10)
7. Bạn có bị chặn (blocked) thường xuyên không? (Hiếm / Thỉnh thoảng / Thường xuyên)

## C — Communication
8. Docs nội bộ đủ không để tự giải quyết vấn đề? (1–5)
9. Bao nhiêu cuộc họp/ngày không cần thiết? (0 / 1 / 2+ )

## E — Efficiency
10. Bạn có flow state (tập trung sâu >2h) bao nhiêu ngày/tuần? (0-1 / 2-3 / 4-5)
11. Điều gì gây gián đoạn nhiều nhất? (CI chậm / môi trường lỗi / thiếu docs / meeting)
EOF

git add dx-survey-template.md && git commit -m "feat: add DX survey SPACE template"

▶️ Bước 2 — Service Quality Scorecard script:

cat > scorecard.py <<'EOF'
#!/usr/bin/env python3
"""Service quality scorecard based on catalog-info.yaml metadata."""
import yaml, json, sys, os, glob

CRITERIA = {
    "has_owner":        ("spec.owner",          10, "Owner defined"),
    "has_lifecycle":    ("spec.lifecycle",       10, "Lifecycle set"),
    "has_system":       ("spec.system",          10, "System assigned"),
    "has_dependencies": ("spec.dependsOn",       10, "Dependencies declared"),
    "has_runbook":      ("metadata.links",       15, "Runbook link present"),
    "has_description":  ("metadata.description", 10, "Description written"),
    "has_tags":         ("metadata.tags",        10, "Tags present"),
    "has_annotations":  ("metadata.annotations", 15, "Annotations/CI present"),
    "is_production":    ("spec.lifecycle",       10, "Is production service"),
}

def get_nested(d, path):
    """Safely get nested dict value by dot-path."""
    for key in path.split("."):
        if not isinstance(d, dict):
            return None
        d = d.get(key)
    return d

def score_service(filepath):
    with open(filepath) as f:
        doc = yaml.safe_load(f)
    if not doc or doc.get("kind") != "Component":
        return None

    name = get_nested(doc, "metadata.name") or filepath
    total, earned = 0, 0
    details = []

    for _key, (path, weight, label) in CRITERIA.items():
        total += weight
        val = get_nested(doc, path)
        passed = bool(val)
        if _key == "is_production":
            passed = val == "production"
        earned += weight if passed else 0
        details.append({"check": label, "weight": weight, "passed": passed})

    score = round(earned / total * 100)
    grade = "A" if score >= 90 else "B" if score >= 75 else "C" if score >= 60 else "D"
    return {"service": name, "score": score, "grade": grade, "details": details}

# Scan catalog YAML files
results = []
for f in glob.glob("platform/catalog/*.yaml") + glob.glob("**/catalog-info.yaml", recursive=True):
    r = score_service(f)
    if r:
        results.append(r)

if not results:
    print("No Component catalog-info.yaml found. Run from repo root.")
    sys.exit(0)

print("\n=== SERVICE QUALITY SCORECARD ===\n")
for r in sorted(results, key=lambda x: -x["score"]):
    print(f"  [{r['grade']}] {r['service']:30s}  {r['score']:3d}/100")
    for d in r["details"]:
        status = "✓" if d["passed"] else "✗"
        print(f"      {status} {d['check']} (+{d['weight']})")
    print()

avg = sum(r["score"] for r in results) / len(results)
print(f"Average score: {avg:.0f}/100")
print("\nImprovements needed:")
for r in results:
    gaps = [d["check"] for d in r["details"] if not d["passed"]]
    if gaps:
        print(f"  {r['service']}: {', '.join(gaps)}")
EOF

python3 scorecard.py

▶️ Bước 3 — Improvement Plan template:

cat > improvement-plan.md <<'EOF'
# Platform DX Improvement Plan — Q3 2026

## Current State (từ survey + scorecard)
- DX NPS nội bộ: 32 (target: 50)
- Thời gian onboard service mới: 3 ngày (target: <4 giờ)
- CI pipeline p50 duration: 18 phút (target: <10 phút)
- Tag coverage: 71% (target: 95%)
- Service với runbook link: 40% (target: 80%)

## OKRs Q3 2026

### Objective: Tăng tốc developer onboarding 10x
- KR1: Golden path template được dùng cho 100% service mới
- KR2: Thời gian scaffold-to-first-deploy <4 giờ
- KR3: Developer satisfaction score ≥4.0/5.0

### Objective: Cải thiện platform reliability
- KR1: CI p50 <10 phút (hiện 18 phút)
- KR2: Self-service provisioning <5 phút (hiện manual ticket 2 ngày)
- KR3: Scorecard average ≥80/100 (hiện 62/100)

## Action Items
| Action                          | Owner        | Due     | Priority |
|---------------------------------|--------------|---------|----------|
| Launch golden path template     | platform-team| 2026-07 | P0       |
| Crossplane XRD cho DB + bucket  | platform-team| 2026-07 | P0       |
| CI cache optimization           | devex-team   | 2026-06 | P1       |
| Backstage catalog import        | all teams    | 2026-08 | P1       |
| Runbook requirement in scorecard| platform-team| 2026-06 | P2       |
EOF

git add scorecard.py improvement-plan.md
git commit -m "feat: add service scorecard script and DX improvement plan"

✅ Kết quả mong đợi: python3 scorecard.py in bảng scorecard với grade A/B/C/D cho từng service, liệt kê improvement gaps. improvement-plan.md có OKR đo được với deadline rõ ràng. Survey template bao phủ đủ 5 chiều SPACE.

🧹 Cleanup: git push origin main — giữ toàn bộ artifacts làm portfolio Platform Engineering.