Test work

Add Terraform Module Development Standards & Enhanced Azure Tests

Summary of Changes

This update introduces a comprehensive set of Terraform module development standards and enhances the testing framework for Azure Terraform modules by introducing new testing scripts, configurations, and extensive documentation. Additionally, environment variables and test workflows have been modified to introduce better defaults and improved validation.

Key Modifications and Their Purpose

1. Documentation and Standards for Terraform Development

• File Added: infra/modules/.cursor/rules/terraform-module.mdc
  • Introduces Terraform module development guidelines, including:
    • Directory structure standards (main.tf, variables.tf, outputs.tf, etc.).
    • Coding conventions for variables, outputs, and dynamic blocks.
    • Testing standards with a focus on both unit and integration tests.
    • Best practices for security, naming, resource management, and CI/CD pipelines.

2. Enhancements to magefile.go

• Added a default setup for RESOURCE_GROUP_NAME in the FindAndRunTests function if not already provided.
• Introduced a Test(module string) function for executing module-specific test.sh scripts.
  • Ensures scripts are executable and validated.
  • Sets default environment variables like RESOURCE_GROUP_NAME and LOCATION if not set.

3. Addition of Module Testing Scripts

• New Files Added: Testing scripts for Azure modules, including:
  • infra/modules/providers/azure/app-insights/test.sh
  • infra/modules/providers/azure/log-analytics/test.sh
  • infra/modules/providers/azure/resource-group/test.sh
  • infra/modules/providers/azure/storage-account/test.sh
  • Purpose: Standardizes tests for core modules, including validation, variable setup, Terraform execution, and cleanup processes.
  • All scripts initialize environment variables, validate configurations, and utilize a shared testing framework for consistency.

4. Updates to Individual Module Testing Configurations

• Introduced tailored testing/main.tf and unit_test.go for the following modules:
  • Azure App Insights:
    • Adds Log Analytics workspace integration with workspace ID validation.
  • Azure Resource Group:
    • Modularized main.tf with variable-driven configurations for name, location, and tagging.
    • Enhanced unit_test.go to utilize dynamic generation of resource group names and better support environment variable overrides.
  • Azure Storage Account:
    • Modularized Terraform configuration with support for additional resource types (containers, queues, and shares).
    • Enhanced tests to validate storage account configurations, including TLS enforcement and HTTPS-only access.

5. Shared Common Testing Functions

• New File Added: infra/modules/providers/azure/test-functions.sh
  • Acts as a base testing library with reusable functions:
    • setup_configuration: Handles common Terraform setup for tests.
    • generate_unique_name: Provides unique name generation for resources.
    • terraform_init_and_apply: Standardizes Terraform initialization and deployment.
    • run_standard_test_sequence: Provides a structured flow for unit and integration tests.
    • Includes environment validation (validate_azure_credentials), cleanup, logging, and testing utilities.

6. Updates to Integration and Unit Tests

• Adjustments to unit_test.go and integration tests of specific modules:
  • Azure Storage Account:
    • ExpectedResourceAttributeValues improved to validate additional outputs like min_tls_version and enable_https_traffic_only.
    • Added dynamic name setups from environment variables for better test control.
  • Azure Resource Group:
    • unit_test.go updated to dynamically pull LOCATION and RESOURCE_GROUP_NAME from the environment or generate them as needed.
  • Integration Test Updates:
    • Refactored options for both unit and integration testing to include explicit parameterization of variables like location, replication_type, etc.

7. Licensing Header Fixes

• Resolved inconsistencies in license headers of Go files (unit_test.go).
  • Reformatted headers for compliance with Apache 2.0 licensing.

8. Improved Azure Module Outputs

• Enhanced outputs.tf for resource-group and storage-account modules:
  • Added structured outputs for resource IDs, properties, and additional sensitive attributes.
  • Enabled detailed output structures for downstream module dependencies.

9. Adjustments to Terraform Providers

• Updated main.tf files for modules (storage-account, resource-group, etc.) to explicitly pin the azurerm provider version (3.90.0).

10. Minor Enhancements to Unit Test Configurations

• Updated tf_options.go to support more flexible integration testing:
  • Added ContainerName, replication_type, and improved handling of environment overrides.

Notable Technical Details

• Dynamic Variable Validation:
  • All module-specific tests employ dynamic resource generation, ensuring tests remain idempotent.
  • Introduced safeguards for environment variable validation and resource naming conventions.
• Integrated Resource Cleanup:
  • test.sh scripts handle thorough cleanup via Azure CLI (az group delete), ensuring no residual artifacts post-tests.
• Terraform Configuration Validation:
  • Introduces terraform plan as part of the test lifecycle to pre-validate configurations before applying changes.
• Enhanced Testing Framework:
  • Shifts to a unified testing framework (test-functions.sh) to eliminate redundancies across module configurations.
  • Modular composition of tests improves scalability and testing consistency across the infrastructure codebase.