Validation Methodology & Reference Implementation

Executive Summary

This document defines the verification and validation (V&V) methodology applied to the HAPIS Link Budget API and its associated products (SaaS Dashboard, NTN Simulator). It is written for the RF engineer who needs to evaluate HAPIS against alternatives — STK, MATLAB Satellite Communications Toolbox, open-source Python libraries, and internal spreadsheet implementations — and decide whether HAPIS outputs can be trusted as the basis for engineering decisions.

The methodology is structured as a six-tier validation pyramid, adapted from ANSI/ASME V&V 20, NASA-STD-7009A, and ITU-R P.311. Every implemented model is verified against its source specification and (where reference data is published) validated against the worked examples of that specification. Cross-tool comparison, empirical measurement comparison, and formal sensitivity analysis are scheduled for upcoming releases; this draft is explicit about which tiers are operational today and which are roadmap.

Stated accuracy: ±0.5 dB against ITU-R reference case studies for atmospheric loss components in their valid parameter ranges. Known limitations and parameter envelopes are documented in § 7.

1. Introduction

1.1 Purpose

The purpose of this document is to provide a transparent, auditable description of how HAPIS validates the correctness of its RF propagation and link budget calculations. It serves three audiences:

1. Engineering reviewers evaluating HAPIS for adoption in a project requiring traceable, defensible RF analysis.
2. Regulatory and certification bodies assessing whether HAPIS-derived analyses meet documentation standards for spectrum and aviation filings.
3. Internal developers maintaining HAPIS, who require a stable specification of validation obligations across versions.

1.2 Scope

This methodology applies to all RF computation modules of HAPIS, specifically:

• Free-space path loss (FSPL)
• Atmospheric gaseous absorption (oxygen and water vapour)
• Rain attenuation (specific and statistical)
• Cloud and fog attenuation
• Clutter loss for terrestrial-component links
• Geometry (slant range, elevation, ground footprint) for stratospheric platforms
• 3GPP NTN reference scenario calculations (link budget, Doppler, propagation delay)

Out of scope for this version: ionospheric scintillation models, multipath / fading time-series synthesis, and end-to-end protocol-level simulation (modelled separately in the NTN Simulator product, validated under its own document).

1.3 Terminology

2. Validation Philosophy

HAPIS is built on three commitments that shape the entire validation methodology:

(1) Traceability over opacity. Every numerical output of the API is computable from a published equation in a named, versioned standard. The mapping between API response fields and source standards is documented and machine-readable. No proprietary corrections or unpublished approximations are applied to ITU-R or 3GPP models without explicit disclosure.

(2) Transparency over marketing claims. Where HAPIS deviates from a reference (e.g., a numerical implementation choice for an integral, a piecewise-linear interpolation between tabulated values, an out-of-range extrapolation), the deviation is documented in the source code, in this whitepaper, and in the validation log. Honest reporting of edge-case deviations is treated as more valuable than uniform “passing” claims.

(3) Reproducibility over assertion. The intent is that any engineer can independently reproduce every numerical claim in this document. § 8 describes which artefacts are reproducible today and which require contacting HAPIS for the underlying test outputs.

3. The HAPIS Validation Pyramid

The validation methodology is organized into six tiers, each addressing a different class of risk. Lower tiers are necessary preconditions for higher tiers: a model that fails Tier 1 cannot be trusted in Tier 3, regardless of how favourable the comparison looks.

                          +---------------------------------+
                          |  TIER 6 - Continuous Validation |  Implemented
                          +---------------------------------+
                          |  TIER 5 - Sensitivity & UQ      |  Planned (v2.0)
                          +---------------------------------+
                          |  TIER 4 - Empirical Validation  |  Planned (v1.1)
                          +---------------------------------+
                          |  TIER 3 - Code-to-Code          |  Planned (v1.1)
                          +---------------------------------+
                          |  TIER 2 - Reference Cases       |  Partial
                          +---------------------------------+
                          |  TIER 1 - Code Verification     |  Implemented
                          +---------------------------------+
                

3.1 Tier 1 — Code Verification Implemented

Question answered: Does the implementation faithfully reproduce the mathematical specification of the source standard?

Methods applied today:

• Formula-by-formula unit tests. Each equation in each source standard (ITU-R P.525 Eq. 1, P.676 Annex 1 slant-path integration, P.838-3 specific-attenuation coefficients, etc.) is implemented in its own pure function and tested independently with hand-computed input/output pairs derived directly from the equation.
• Edge-case tests. Boundary conditions: zero distance, zero frequency, 90° elevation (zenith), 0° elevation (horizontal), maximum specified rain rate, dry atmosphere, fully saturated atmosphere.
• Numerical stability tests. Inputs that approach singularities (e.g., 0° elevation in slant-path integrals) are tested for graceful degradation and explicit error messaging rather than silent NaN propagation.
• Branch coverage. A 70 % coverage floor for the api and hapis_core packages is enforced in CI via pytest --cov-fail-under=70.

Planned for v1.1: property-based testing via hypothesis for functions where exhaustive testing is infeasible (monotonicity, symmetry, non-negativity, continuity over wide input domains).

Current deliverable: 175 product unit tests in the engine repository, with 100 % pass rate required for any release.

3.2 Tier 2 — Reference Case Validation Partial

Question answered: Does HAPIS reproduce the worked examples published in the source standards themselves?

ITU-R and 3GPP recommendations frequently include annex tables of “test results” or “example calculations” intended exactly for this purpose. These are the closest thing to ground truth that exists for these models.

Methods applied:

• For each implemented standard, a representative selection of worked examples and tabulated test values from the official annex is reproduced by HAPIS.
• Deviations are reported per case; aggregate statistics (mean bias, RMSE, max deviation, percentage within tolerance) will be added as the case catalogue is expanded.
• Tolerances are defined per standard (see § 6.2).

Reference sources currently covered:

Planned for v1.1: a full per-standard reference-case report (input/output tables, deviation statistics) published as a downloadable appendix.

3.3 Tier 3 — Code-to-Code Cross-Validation Planned (v1.1)

Question answered: Do independent implementations of the same standards converge on the same numerical answer?

This is the tier most relevant to the RF engineer comparing HAPIS to existing tools. It addresses both implementation bugs in HAPIS and ambiguities in the source standards themselves. This tier is not yet operational — cross-validation runs against the reference tools listed below are scheduled for HAPIS Engine v1.1.

Reference tools queued for cross-validation:

Planned procedure: for each of 50+ canonical scenarios spanning frequency (1–100 GHz), elevation (5°–90°), platform altitude (terrestrial, HAPS 20 km, LEO, GEO), and atmospheric conditions (dry / standard / tropical, no-rain / light / heavy), HAPIS outputs will be compared head-to-head against the reference tools. Deviations will be tabulated and the matrix published as a public artefact with v1.1.

3.4 Tier 4 — Empirical Measurement Validation Planned (v1.1)

Question answered: Does the model — as implemented in HAPIS — predict physical reality within the documented tolerance?

This tier is where most RF tools quietly stop. HAPIS does not intend to stop here, but it is honest that empirical-validation runs are not yet operational. The datasets and methodology below describe the planned approach for v1.1.

Public measurement datasets targeted for the v1.1 empirical-validation run:

Planned methodology: HAPIS predictions computed for the same site / frequency / exceedance probability combinations covered by each dataset; statistical comparison of predicted vs. measured attenuation CDF, bias, RMSE across exceedance levels (0.001 %, 0.01 %, 0.1 %, 1 %); reporting per ITU-R P.311 methodology.

Honest disclosure: HAPS-specific empirical data (stratospheric platforms at 20 km) is sparse in the open literature. v1.1 will validate against (a) the equivalent slant-path geometry from satellite measurements adapted to 20 km altitude, and (b) where available, published HAPS field-trial data. This limitation is reiterated in § 7.

3.5 Tier 5 — Sensitivity and Uncertainty Quantification Planned (v2.0)

Question answered: How sensitive are HAPIS outputs to input uncertainty, and what is the resulting confidence interval on results?

A link-budget output of “10.66 dB margin” is meaningless without knowing whether the underlying inputs (rain rate, antenna gain, T_sys) could shift that figure by ±0.5 dB or ±5 dB.

Planned methods (v2.0):

• One-at-a-time (OAT) sensitivity analysis for each top-level link-budget output, varying each input across its physically plausible range.
• Monte Carlo uncertainty propagation with documented input distributions (e.g., rain rate from ITU-R P.837 exceedance distributions, antenna gain ±0.5 dB, T_sys ±10 K).
• Sobol global sensitivity indices for the most operationally relevant outputs (link margin, coverage radius), identifying which inputs dominate uncertainty.
• Output: confidence intervals (5th, 50th, 95th percentile) on link margin for each operational scenario.

Why this matters competitively: most commercial tools return a single number. HAPIS plans to optionally return a number with a defensible uncertainty bound — directly usable in BVLOS safety cases, feasibility-study risk sections, and regulatory filings.

3.6 Tier 6 — Continuous Validation Implemented

Question answered: Does HAPIS remain valid across releases, dependency updates, and platform changes?

A model that was correct at v1.0 and silently regressed by v1.3 is worse than no model. HAPIS runs a GitHub Actions pipeline on every push and pull request to the main branch (defined in .github/workflows/ci.yml) with the following jobs:

Any deviation in test results, coverage drop below 70 %, or new high-severity CVE blocks the build. The CI status is visible in the engine's GitHub Actions tab.

Planned for v1.1: baseline outputs for reference cases version-controlled so that numerical drift triggers a flagged PR review even when within tolerance; semantic-versioning policy and machine-readable validation manifest published per release.

4. Per-Standard Validation Plan

This section details how each standard is validated under the framework above. Status badges indicate which tiers are operational today for that standard.

4.1 ITU-R P.525 — Free-Space Path Loss

Implemented equation (P.525-4 Eq. 1):

L_bf = 20 log10(4πd/λ) = 32.44 + 20 log10(f_MHz) + 20 log10(d_km)

Tolerance: ±0.01 dB (numerical round-off). This is closed-form arithmetic; any deviation beyond floating-point round-off indicates a bug.

Validation today: Tier 1 unit tests at 1 GHz, 14 GHz, 28 GHz, 40 GHz, 100 GHz across 100 km, 1200 km, 31 km (HAPS slant), 36 000 km (GEO slant); Tier 2 P.525 Annex 1 worked-example reproduction.

Planned (v1.1): Tier 3 cross-check vs. symbolic computation, STK, MATLAB.

4.2 ITU-R P.676 — Atmospheric Gaseous Absorption

Implemented: P.676 oxygen and water-vapour line absorption plus the slant-path integration using exponential atmosphere model. The current implementation prioritizes accuracy in the operational HAPS bands (S, Ka, Q, V).

Tolerance: ±0.3 dB for zenith attenuation; ±0.5 dB for slant paths at elevation ≥10°. Larger deviations at low elevation (<5°) are documented and flagged.

Validation today: Tier 1 unit tests per equation; Tier 2 against P.676 Annex 1 zenith attenuation values at 22.235 GHz (water-vapour line), 50 GHz, 60 GHz (oxygen complex).

Planned (v1.1): Tier 3 cross-validation against itur.models.itu676 and MATLAB gaspl(). Planned (v1.1): Tier 4 comparison against radiometric data from ITU-R DBSG5.

4.3 ITU-R P.838-3 — Rain Attenuation

Implemented: P.838-3 specific attenuation γ_R = k R^α with polarization-dependent coefficients k_H, α_H, k_V, α_V and elevation- / polarization-tilt corrections.

Tolerance: ±0.5 dB for the specific attenuation; statistical slant-path attenuation tolerances grow at low exceedance probabilities (acknowledged limitation of the underlying model, not HAPIS).

Validation today: Tier 1 coefficient interpolation tested against P.838-3 Table 1; Tier 2 worked-example reproduction.

Planned (v1.1): Tier 3 cross-validation against itur.models.itu838 and MATLAB rainpl(); Tier 4 ITU-R DBSG3 rain attenuation CDF comparison at multiple sites.

4.4 ITU-R P.840 — Cloud and Fog Attenuation

Implemented: P.840 Rayleigh approximation for cloud specific attenuation, integrated over columnar liquid water content. Discrete cloud-type presets (clear, cumulus, stratus, nimbostratus) mapped to canonical LWC values.

Tolerance: ±0.3 dB at frequencies ≤100 GHz.

Validation today: Tier 1 specific-attenuation coefficient K_l verified at multiple frequencies and cloud temperatures; Tier 2 P.840 Annex 1 example reproduction.

Planned (v1.1): Tier 3 cross-validation against itur.models.itu840.

4.5 ITU-R P.2108 — Clutter Loss

Implemented: P.2108-1 Earth-space clutter model for the terrestrial component of HAPS user-terminal links.

Tolerance: ±1.0 dB. Note: P.2108 is a statistical model with intrinsic spread; tolerance reflects the model's own uncertainty, not implementation error.

Validation today: Tier 1 implementation against P.2108-1 Eq. 4; Tier 2 reproduction of P.2108-1 reference curves.

4.6 3GPP TR 38.811 / 38.821 / 38.822 — NTN Reference Scenarios

Implemented: TR 38.811 link-budget composition for HAPS (20 km) and reference NTN scenarios. Doppler shift and Doppler rate per § 6.3. Propagation delay per § 6.4.

Tolerance: each individual loss term within its standard-specific tolerance (above). Aggregated link budget within ±0.5 dB of TR 38.811 published reference budget tables for HAPS scenarios.

Validation today: Tier 2 partial reproduction of TR 38.811 § 6.6 HAPS reference scenario.

Planned (v1.1): Tier 2 full reproduction of TR 38.821 § 6.1.1 link-budget reference cases; Tier 3 cross-validation against MATLAB Satellite Communications Toolbox satelliteScenario.

Disclosure: TR 38.822 is referenced as a draft specification. The whitepaper and validation log mark all 38.822-derived requirements as draft and subject to change.

5. Competitive Positioning — Honest Comparison

This section is provided not as marketing but as the comparison that any technically literate prospect will perform on their own. It is more useful for HAPIS to publish it honestly than to leave the prospect to construct an unfavourable version.

Where HAPIS does not yet compete:

• Orbital mechanics depth. STK remains the leader for high-fidelity orbital propagation, sensor pointing, and mission-design workflows. HAPIS does not attempt to replace SGP4 / numerical-integrator-class orbital modelling.
• Antenna-array beamforming simulation. STK and MATLAB Phased Array Toolbox provide capabilities HAPIS does not.
• Full physical-layer simulation (waveform-level). GNU Radio and MATLAB 5G / Comm Toolboxes are the appropriate tools for waveform-level link emulation; HAPIS operates at the link-budget level.

Honest disclosure of where HAPIS does not compete is a deliberate trust signal. An RF engineer who asks “but what about X?” and finds X already acknowledged in the methodology document treats the rest of the methodology with proportionally more confidence.

6. Metrics and Reporting Format

6.1 Statistical Metrics

For every cross-validation and empirical comparison (operational today for Tiers 1–2; planned for Tiers 3–4), HAPIS reports:

6.2 Per-Standard Tolerance Table

6.3 Public Reporting

Today, the HAPIS validation surface is:

• This methodology document.
• The engine's GitHub Actions CI status (lint, test + coverage, security scan, signed image with SBOM) for releases tagged on the main branch.
• Test-output artefacts available to engineering reviewers on request to contact@hapintel.com.

Planned for v1.1: a downloadable validation report (PDF + machine-readable manifest) per release, with the Tier 2 reference-case catalogue and the Tier 3 cross-validation matrix as appendices.

7. Known Limitations and Future Work

A whitepaper that omits limitations is a marketing document. The following are explicit limitations of HAPIS as of v1.0:

7.1 Model limitations inherited from source standards

• ITU-R P.618 rain attenuation at exceedance probabilities below 0.001 % has intrinsic uncertainty larger than ±1 dB; HAPIS does not improve on this.
• ITU-R P.676 line-by-line model is most accurate for clear sky; in heavy precipitation, gas + rain interaction is approximated as additive.
• ITU-R P.840 cloud attenuation uses Rayleigh approximation, valid for cloud droplets up to ~100 µm — not applicable to ice clouds at very high frequencies.

7.2 HAPS-specific empirical data scarcity

Stratospheric platforms at 18–25 km altitude lack the extensive multi-year beacon measurement campaigns available for GEO and LEO satellites. The v1.1 empirical-validation cycle will rely on (a) the geometric adaptation of slant-path satellite measurements to 20 km altitude and (b) the limited published HAPS field-trial data. Operationally observed HAPS performance may differ from predictions by amounts not yet bounded by published data.

7.3 Out-of-envelope inputs

The API rejects requests outside the validity envelope of the underlying standards (e.g., elevation <5° for some atmospheric models, frequencies outside 1–1000 GHz for P.676). Where the envelope is soft, the API returns the result with a warning flag indicating reduced confidence.

7.4 Not modelled in v1.0

• Ionospheric scintillation
• Tropospheric scintillation time series (P.618 mean scintillation only)
• Faraday rotation (relevant only at sub-GHz frequencies, outside HAPS primary band)
• Multipath-fading time-series synthesis (handled separately by the NTN Simulator product)

These are scheduled for v2.0 and explicitly out of scope for this validation cycle.

8. Reproducibility — How to Verify Claims in This Document

Reproducible today, without contacting HAPIS:

• The closed-form FSPL values in Appendix A § A.1 (a hand calculation per P.525-4 Eq. 1 reproduces the table).
• The CI configuration described in Tier 6 (visible in the engine repository's .github/workflows/ci.yml for licensees and engineering reviewers with access).

Available on request to engineering reviewers (contact@hapintel.com):

• Full Tier 1 test outputs (175 unit tests, pass / fail / numerical deltas).
• Tier 2 reference-case logs (which ITU-R annex examples are reproduced, with input / output / tolerance).
• Coverage report from the most recent CI run.

Planned with v1.1: a self-contained reproducibility bundle (script + reference inputs + expected outputs + comparison templates for STK / MATLAB / itur / pycraf) published as a public artefact. Until that bundle is published, the on-request channel above is the canonical path.

9. Versioning and Document Lifecycle

This methodology document is versioned with the HAPIS product. Material changes (new standards added, tolerance revisions, validation suite restructuring) trigger a new whitepaper version. The version table below identifies the applicable HAPIS API version range.

10. References

ITU-R Recommendations

1. ITU-R Rec. P.525-4, “Calculation of free-space attenuation,” 2019.
2. ITU-R Rec. P.618-13, “Propagation data and prediction methods required for the design of Earth-space telecommunication systems,” 2017.
3. ITU-R Rec. P.676-13, “Attenuation by atmospheric gases and related effects,” 2022.
4. ITU-R Rec. P.838-3, “Specific attenuation model for rain for use in prediction methods,” 2005.
5. ITU-R Rec. P.840-8, “Attenuation due to clouds and fog,” 2019.
6. ITU-R Rec. P.2108-1, “Prediction of clutter loss,” 2021.
7. ITU-R Rec. P.311-19, “Acquisition, presentation and analysis of data in studies of radiowave propagation,” 2021.
8. ITU-R Rec. P.837-7, “Characteristics of precipitation for propagation modelling,” 2017.

3GPP Technical Reports

9. 3GPP TR 38.811 v15.4.0, “Study on New Radio (NR) to support non-terrestrial networks,” 2020.
10. 3GPP TR 38.821 v16.2.0, “Solutions for NR to support non-terrestrial networks (NTN),” 2023.
11. 3GPP TR 38.822 (draft), “NR support for high-altitude platform station (HAPS),” 2021.

V&V and Methodology Standards

12. ANSI/ASME V&V 20-2009, “Standard for verification and validation in computational fluid dynamics and heat transfer.”
13. NASA-STD-7009A, “Standard for models and simulations,” 2016.
14. IEEE Std 1012-2016, “IEEE standard for system, software, and hardware verification and validation.”

Textbooks

15. B. Sklar, Digital Communications: Fundamentals and Applications, 2nd ed., Prentice Hall, 2001.
16. T. Pratt, C. W. Bostian, and J. E. Allnutt, Satellite Communications, 3rd ed., Wiley, 2019.
17. G. Maral, M. Bousquet, and Z. Sun, Satellite Communications Systems, 6th ed., Wiley, 2020.
18. S. R. Saunders and A. Aragón-Zavala, Antennas and Propagation for Wireless Communication Systems, 2nd ed., Wiley, 2007.

Measurement Campaigns

19. A. D. Panagopoulos et al., “Satellite communications at Ku, Ka, and V bands: Propagation impairments and mitigation techniques,” IEEE Commun. Surveys Tuts., 2004.
20. C. Riva et al., “The propagation experiment with the ALPHASAT TDP5 Q/V-band payload,” 9th EuCAP Conf., 2015.
21. J. P. V. Poiares Baptista, “OPEX (Olympus Propagation EXperiment) measurements,” ESA, 1994.

Open-Source Reference Implementations

22. I. del Portillo, “ITU-Rpy: Python implementation of ITU-R recommendations for atmospheric propagation.”
23. B. Winkel et al., “pycraf: Spectrum-management compatibility studies in Python.”

Appendix A — Sample Test Case Catalog

A representative subset of the validation case catalogue is reproduced here for inspection. The full catalogue will accompany v1.1.

A.1 FSPL test cases

Values computed via P.525-4 Eq. 1 (constant 32.44; floating-point round-off accounts for sub-0.01 dB differences against alternative published constants such as 32.45).

A.2 Gas attenuation test cases (zenith)

Indicative reference values from ITU-R P.676-13 Annex 1 worked tables, mid-latitude summer standard atmosphere. Full per-frequency table planned with v1.1.

A.3 Rain attenuation test cases (P.838-3 specific)

Specific attenuation γ_R = k R^α, vertical polarization, computed from the P.838-3 tabulated coefficients.

A.4 HAPS reference scenario (3GPP TR 38.811 § 6.6, HAPS 20 km)

Appendix B — Glossary