🧱 Security & Administration

Unity Catalog security, Secrets, Network isolation, Cluster policies

Databricks Data Engineer · DE-Associate · ⏱️ 20 min · ❓ 4 savol

🔐 Unity Catalog Security

GRANT/REVOKE — SQL orqali: `GRANT SELECT ON TABLE catalog.schema.table TO group`. Row-level security — row filter orqali. Column-level masking — masking policy. Data lineage — ma'lumot qayerdan keldi, qayerga ketdi. Audit Log — kim qaysi dataga kirdi.

🔑 Secrets Management

Databricks Secrets — API key, DB password kabi sensitive ma'lumot saqlash. Secret Scope: Databricks-backed yoki Azure Key Vault / AWS Secrets Manager backed. `dbutils.secrets.get(scope, key)` — kod ichida. Secret value log da yashiriladi. CLI: `databricks secrets put-secret`.

🌐 Network Isolation

VNet/VPC Injection — Databricks cluster larini customer VPC/VNet da ishlatish (Customer-Managed VPC). Private Link — Databricks control plane bilan private connection. IP Access List — Workspace ga faqat ma'lum IP lardan kirish. No Public IP (NPIP) — cluster node lari public IP siz.

⚙️ Cluster Policies & Cost Management

Cluster Policy — admin tomonidan cheklangan konfiguratsiya (max cluster size, instance type, timeout). Non-admin userlar policy ga rioya qilishi shart. Cost tagging — cluster va job larga tag qo'yish (department, project). Budget alert — Databricks Account Console da.

💡 Asosiy nuqtalar

Unity Catalog: GRANT/REVOKE, row filter, column masking, lineage
Secrets: dbutils.secrets.get() — log da yashirilgan
Secret Scope: Databricks-backed yoki cloud KV backed
VPC Injection: customer VPC da cluster
Cluster Policy: admin cheklaydi, user rioya qiladi

📋 Kod misoli

# Secrets
# CLI: databricks secrets create-scope --scope my-scope
# CLI: databricks secrets put-secret --scope my-scope --key db-password --string-value "secret123"

# Notebook da ishlatish
password = dbutils.secrets.get(scope="my-scope", key="db-password")
# password = "[REDACTED]" — print qilsangiz ham ko'rsatmaydi

# Unity Catalog GRANT
spark.sql("""
  GRANT SELECT ON TABLE main.sales.orders TO `data-analysts`
""")

spark.sql("""
  GRANT MODIFY ON SCHEMA main.sales TO `data-engineers`
""")

# Row-level security (row filter)
spark.sql("""
  CREATE ROW FILTER region_filter ON main.sales.orders
  AS (user_name) RETURN CURRENT_USER() = user_name OR IS_ACCOUNT_GROUP_MEMBER('admin')
""")

spark.sql("""
  ALTER TABLE main.sales.orders SET ROW FILTER region_filter ON (user_name)
""")

🎯 Imtihon maslahatlari

dbutils.secrets.get() — secret value log da yashiriladi (REDACTED ko'rsatiladi)
Row-level security: Unity Catalog row filter — SQL WHERE sharti kabi
No Public IP (NPIP): cluster node lari faqat private IP, NAT Gateway orqali chiqadi
Cluster Policy — admin konfiguratsiyani cheklaydi, non-admin faqat ruxsat etilgan variantlarni tanlaydi
Audit Log: Unity Catalog da kim qaysi tablega kirganini log qiladi

⚠️ Ko'p adashadigan

Secret ni notebook da print qilish xavfsiz deb o'ylash — print ham ko'rsatmaydi (lekin e'tiroz qiling)
VPC Injection = Peering deb o'ylash — VPC Injection cluster VPC DA, Peering esa ikki VPC ORASIDA
Cluster Policy faqat instance type cheklaydi deb o'ylash — timeout, spark version, env var ham cheklash mumkin

🧠 Eslab qolish: "SNPC" = Secrets (dbutils), Network (VPC injection), Policy (cluster), Catalog (Unity) — Databricks security 4 qatlami