Methodology

# Methodology and limitations

## Purpose

Describe how public H-1B Labor Condition Application filings compare against industry
peers. The output is a descriptive `anomaly_score`, not a determination of fraud or any
other violation of law.

## Limitations (read first)

1. **No validation.** The model has never been tested against a labeled set of adjudicated
   violations. There is no measured precision, recall, or error rate. A high score has no
   demonstrated relationship to unlawful conduct.
2. **Relative, not absolute.** Scores rank employers against others in the current extract.
   Re-running on a different quarter changes the distribution.
3. **Business-model confounding.** Filing volume, client-site placement rate, and multi-entity
   structure describe the staffing and consulting business model. Large legitimate staffing
   firms score above zero on these by construction. Weights for these indicators are set low
   deliberately, which mitigates but does not eliminate the effect.
4. **Imperfect entity matching.** The Texas bulk registry contains no federal tax ID, so
   employers are matched by normalized name. This produces both false matches and misses.
   Roughly a quarter of employers match at all; the rest have registry indicators disabled.
5. **Coarse occupation check.** The NAICS-to-SOC consistency test is a lookup of expected
   occupation groups per industry sector and misfires on diversified employers.
6. **No appeal path.** There is no mechanism for a named employer to contest or correct a
   score. This alone means the output must not drive adverse determinations.

## Core rule: missing data is excluded, never scored

Each indicator is normalized to 0–1 and multiplied by its weight. An indicator whose source
data is unavailable for a given employer has `is_present = false` and weight 0. The score is

`weighted_average = (sum(feature_value * weight)) / (sum(weights of present features))`

scaled to 0–100. Confidence is present-weight / loadable-weight (features whose **source
file** is loaded in this run). Unloaded hub, officer, and WHD files are omitted from the
denominator so their absence does not collapse the whole board. Rows with confidence
**below 0.75** are tagged Insufficient Data and held off the anomaly boards.

Consequences:

- An employer with no registry match is not penalized as "not on the roll"; the registry
  features are absent, which usually drops confidence below the 0.75 gate.
- Two employers with different data coverage remain comparable, because each is scored
  against the weight that applied to it.
- `unavailable_features` and `feature_citations` list what could not be evaluated and
  which extract + timestamp produced each fired flag.

### Four boards

| Board | Rule |
|---|---|
| Validated leads | Confidence ≥ 0.75 and corroborated by WHD, a matched USCIS hub mismatch, or a matched thin/inactive franchise record |
| Unverified anomalies | Confidence ≥ 0.75, not institutional, no corroborating source |
| High-volume / institutional | Named household operators, universities, hospitals, public agencies |
| Insufficient data | Confidence < 0.75 (audit only) |

### Failure modes this prevents

| Bug | Symptom before the guard |
|---|---|
| `bool(NaN)` returns `True` | 99.8% of employers received the maximum "inactive registry" penalty, including large public companies with active registrations |
| Blank-string FEIN column passes `notna()` | The registry join selected the tax-ID path and matched 0 of 11,559 rows silently |
| Empty USCIS extract treated as zero petitions | 53% of employers were flagged for "LCAs without petitions" against a table that was never loaded |
| `groupby.transform()` re-keyed through `.map()` | Every NAICS median was NaN, and the UI rendered `nanx` |
| Attorney field populated with first names only | 7,620 unrelated employers (66% of the universe) collapsed into a single "network" |
| Street+ZIP treated as a company | Every tenant in a Regus / office park grouped together |
| Missing franchise join scored as "unregistered" | Real operating companies ranked as shells |

Each has a regression test in `tests/test_missing_data_suppression.py`.

## Indicators

| Feature | Weight | Gate | What it measures |
|---------|--------|------|------------------|
| senior_title_level_i | 18 | — | Senior titles at DOL Wage Level I; 25% weight for NAICS 5112 / 5415 |
| low_wage_pw_ratio | 18 | wage data | Offered wage below prevailing wage |
| thin_footprint | 18 | registry match | Inactive status, no right to transact, or PO Box |
| new_entity_high_volume | 18 | registry match | High volume soon after registration |
| officer_reuse | 18 | officer records | Shared officers across tax IDs |
| prior_whd_violation | 20 | list supplied | Enforcement list match |
| shared_suite | 14 | — | Other H-1B tax IDs at the same suite/unit/floor (not street+ZIP) |
| trade_name_tech_mismatch | 14 | — | Media/events/hospitality legal name with computer or engineering LCAs |
| lca_vs_uscis_mismatch | 12 | hub match | LCA count vs approved petitions |
| wage_underpayment_pct | 10 | wage data | Depth of wage gap |
| naics_soc_mismatch | 8 | — | Occupation uncommon for industry (does not catch a media brand that self-codes as 541511) |
| volume_naics_pctile | 5 | — | Volume percentile (scale proxy) |
| secondary_entity_rate | 4 | — | Client-site placement (normal for staffing; not used for clustering) |
| volume_zscore_naics | 3 | — | Volume z-score (scale proxy) |
| cluster_size | 3 | — | Officers, parent FEIN, or shared suite |

Rate-based indicators are multiplied by `min(1, lca_n / 10)` so small-sample rates cannot
dominate the ranking.

### Dead indicator disclosure

DOL will not certify an LCA whose offered wage falls below the prevailing wage. On
certified-only extracts, `median_wage_pw_ratio` is at or above 1.0 for essentially every
employer, so the wage indicators never fire. They are retained for extracts containing
denials and for supplied enforcement lists, but they contribute nothing to current rankings.

## Peer comparison columns

`volume_naics_pctile`, `volume_zscore_naics_raw`, `volume_vs_naics_median`,
`national_volume_pctile`, `secondary_vs_naics_pctile`, `lca_hub_ratio`,
`median_wage_pw_ratio`, `comparison_summary`. Fields render as `n/a` or `not loaded` when
the underlying source is unavailable rather than defaulting to a numeric value.

## Entity grouping

Connected components over edges from:

- shared parent FEIN (when that column exists on the extract)
- shared officer name with corroborating address or phone
- shared **unit / suite / floor** on the employer HQ or Secretary of State address

Street number + ZIP without a unit is **not** an edge (office-park problem).
Shared attorneys and shared client-site cities are **not** used.

Guards against degenerate merges:

- Suites with more than 8 distinct H-1B FEINs are skipped (virtual-office mills).
- Officer reuse without address or phone is recorded as a weak edge only.

Group membership means the tax IDs share an attribute. It does not imply common ownership,
control, or coordination.

## What filings cannot show

LCA disclosure files do not record:

- Payments for a visa or green-card slot
- Attorney correspondence or threats
- Whether the listed job is real or where the worker actually works
- Worker or recruiter nationality

Those facts have to come from complaints, bank records, or interviews. This model can
only score the public-form patterns listed above.

## Validation plan (not yet performed)

The model should be tested against DOL willful-violator debarments and the WHD violator
list before its rankings are given any weight. The acceptance criterion is straightforward:
**known violators must rank above large ordinary staffing firms.** Until that test is run
and its results published here, the ranking order should be treated as unvalidated.

## Export packages

`out/packages/<group_id>/` contains member entities, underlying LCA rows, and a
`sources.txt` recording exact input filenames and fetch timestamps. Referral or tip drafts
are intentionally not produced.