One-time cloud setup needed before infra/terraform-test and infra/terraform-staging can be applied #31

Activity

1. amar-python

   added

   enhancementNew feature or requestNew feature or request

   on Aug 13, 2026
2. amar-python commented on Aug 13, 2026

   

   amar-python

   on Aug 13, 2026

   OwnerAuthor

   More actions

   Blocked on #32 — az CLI on the dev machine currently can't authenticate (SSL certificate verification failure), so the state-storage creation step below hasn't been done yet. Nothing in this checklist has been actioned; infra/terraform-test and infra/terraform-staging remain code-only with the backend block commented out.
3. amar-python commented on Aug 13, 2026

   

   amar-python

   on Aug 13, 2026

   OwnerAuthor

   More actions

   Still blocked on #32 — the SSL certificate issue turned out to be Norton Antivirus's SSL/TLS scanning feature generating a non-compliant root CA (missing critical Basic Constraints), not a simple missing-from-bundle problem. See #32 for the full diagnosis and what's needed to actually unblock this (a Norton settings change, or running from an unaffected network/Cloud Shell). Nothing in this checklist has been actioned — infra/terraform-test and infra/terraform-staging remain code-only, as they should until this is resolved and reviewed.
4. amar-python commented on Aug 13, 2026

   

   amar-python

   on Aug 13, 2026

   OwnerAuthor

   More actions

   Update: found a second, independent local blocker — #34 (Terraform's own provider-plugin IPC handshake fails locally, unrelated to #32/Azure/Norton — reproduces even with no network calls, and even against the pre-existing terraform-prod stack). So local terraform apply isn't viable right now for two separate reasons, not just #32. Practical path forward is running from Azure Cloud Shell instead of this machine — offered to drive that via browser automation but holding for your go-ahead given it provisions real, billed resources (~AUD 570-770/mo combined for test+staging at default sizing).
5. amar-python commented on Aug 14, 2026

   

   amar-python

   on Aug 14, 2026

   OwnerAuthor

   More actions

   Execution plan — Azure Cloud Shell walkthrough

   Decided approach: since local az/terraform are blocked on this machine (#32, #34 — both traced to Norton's local TLS interception, declined to fix by changing Norton settings), this runs from Azure Cloud Shell instead, driven manually by the repo owner (not automated). Script pushes directly to main (no PR) — a deliberate choice, not an oversight.

   1. Open Azure Cloud Shell

   Go to shell.azure.com (or portal.azure.com → the >_ icon). First run: choose Bash, not PowerShell. If prompted to create a Cloud Shell storage account, accept the default — that's Cloud Shell's own file storage, unrelated to the Terraform state storage the script below creates.

   2. Paste the script

   Click into the terminal, paste with Ctrl+Shift+V (or right-click → Paste; Cmd+V on Mac), then Enter:

   #!/usr/bin/env bash
   set -euo pipefail
   
   REGION="australiaeast"
   
   # ── 0. Clone the repo ─────────────────────────────────────────────────────────
   if [ ! -d PostgreDataMigrationAppWithCSVLoader ]; then
     git clone https://github.com/amar-python/PostgreDataMigrationAppWithCSVLoader.git
   fi
   cd PostgreDataMigrationAppWithCSVLoader
   
   # ── 1. Shared state-storage resource group ────────────────────────────────────
   az group create --name rg-tfstate --location "$REGION"
   
   # ── 2. Per-environment state storage account + container ──────────────────────
   setup_state_storage() {
     local env="$1"
     local suffix
     suffix=$(openssl rand -hex 3)
     local account="sttfstate${env}${suffix}"
   
     az storage account create \
         --name "$account" \
         --resource-group rg-tfstate \
         --location "$REGION" \
         --sku Standard_LRS \
         --encryption-services blob \
         --min-tls-version TLS1_2
   
     az storage container create \
         --account-name "$account" \
         --name tfstate \
         --auth-mode login
   
     echo "$account"
   }
   
   TEST_ACCOUNT=$(setup_state_storage test)
   STAGING_ACCOUNT=$(setup_state_storage staging)
   
   echo "Test storage account:    $TEST_ACCOUNT"
   echo "Staging storage account: $STAGING_ACCOUNT"
   
   # ── 3. Wire the backend "azurerm" block in each main.tf ────────────────────────
   python3 - "$TEST_ACCOUNT" "$STAGING_ACCOUNT" <<'PYEOF'
   import re, sys
   
   test_account, staging_account = sys.argv[1], sys.argv[2]
   
   def wire_backend(path: str, account: str, key: str) -> None:
       text = open(path, encoding="utf-8").read()
       pattern = re.compile(
           r"  #\n"
           r'  # backend "azurerm" \{\n'
           r'  #   resource_group_name  = "rg-tfstate"\n'
           r"  #   storage_account_name = \"[^\"]+\"  # from the one-time setup script\n"
           r'  #   container_name       = "tfstate"\n'
           r'  #   key                  = "[^"]+"\n'
           r"  # \}\n",
       )
       replacement = (
           '  backend "azurerm" {\n'
           '    resource_group_name  = "rg-tfstate"\n'
           f'    storage_account_name = "{account}"\n'
           '    container_name       = "tfstate"\n'
           f'    key                  = "{key}"\n'
           "  }\n"
       )
       new_text, n = pattern.subn(replacement, text)
       if n != 1:
           sys.exit(f"Expected exactly 1 match in {path}, found {n} — check the file wasn't already edited.")
       open(path, "w", encoding="utf-8").write(new_text)
       print(f"Wired backend in {path} -> {account}")
   
   wire_backend("infra/terraform-test/main.tf", test_account, "te-test.tfstate")
   wire_backend("infra/terraform-staging/main.tf", staging_account, "te-staging.tfstate")
   PYEOF
   
   # ── 4. Commit and push the wired backend blocks to main ────────────────────────
   git add infra/terraform-test/main.tf infra/terraform-staging/main.tf
   git commit -m "Wire remote state backend for test/staging (issue #31)"
   git push
   
   # ── 5. init -> plan -> apply, per environment ───────────────────────────────────
   run_environment() {
     local env_dir="$1"
     echo
     echo "════════════════════════════════════════════════════════════════"
     echo " $env_dir"
     echo "════════════════════════════════════════════════════════════════"
     ( cd "$env_dir" && terraform init )
     ( cd "$env_dir" && terraform plan )
     echo
     echo ">>> Review the plan above carefully before continuing."
     echo ">>> terraform apply will now show the plan again and prompt you to"
     echo ">>> type 'yes' — it will NOT create anything until you do."
     echo
     ( cd "$env_dir" && terraform apply )
   }
   
   run_environment infra/terraform-test
   run_environment infra/terraform-staging
   
   echo
   echo "Done. Outputs:"
   ( cd infra/terraform-test && terraform output )
   ( cd infra/terraform-staging && terraform output )

   3. What happens, and where to act

   1. Repo clones — a few seconds. If the repo is private and Cloud Shell isn't already authenticated to GitHub, git clone/git push prompt for a device-login flow or a PAT.
   2. Two storage accounts get created (az storage account create × 2) — ~1 minute each, no interaction.
   3. Backend blocks wired + pushed to main — automatic, unless git auth is needed (see above).
   4. For terraform-test, then terraform-staging, in turn:
      • terraform init — downloads providers, ~1 minute.
      • terraform plan — prints everything about to be created. Read it.
      • terraform apply — prints the plan again, then stops and prompts:

        Do you want to perform these actions?
          Enter a value:
        

        Must type yes to actually provision. Anything else cancels — no -auto-approve anywhere in this script, on purpose.
      • Provisioning itself: ~10-15+ minutes per environment (Flexible Server + private endpoints are the slow part).
   5. Repeats for staging.
   6. Prints outputs — resource group names, DB hostnames, etc. — at the end.

   4. If something goes wrong

   It's safe to close the Cloud Shell tab and resume later — nothing here is destroyed by closing the window (state lives in the storage accounts from step 2). Re-open Cloud Shell, cd PostgreDataMigrationAppWithCSVLoader, re-run whichever step was left off.

   Cost reminder

   Full apply for both environments provisions real, billed Azure resources — realistically ~AUD 570-780/mo combined for test+staging at the default (Prod-tier) sizing set in each terraform.tfvars.example. Review the terraform plan output for each environment before typing yes.