Adding Mapping documents - #305
grant-minor-sntialtech wants to merge 4 commits into
Conversation
There was a problem hiding this comment.
🟡 Changes recommended
Several mappings, source references, and analysis counts remain inconsistent or unsupported.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Adds paired summary and verbose source-to-PDC mapping analyses for TechSoup OKF, Grantmakers.io, Cuoree Mani, and Charity Navigator.
Changes:
- Documents field coverage, gaps, and data-quality concerns.
- Maps external fields to PDC fields and notes modeling limitations.
- Provides detailed analyses intended for AI ingestion.
File summaries
| File | Description |
|---|---|
docs/mappings/techsoup-okf-field-gap-analysis.verbose.md |
Detailed TechSoup analysis |
docs/mappings/techsoup-okf-field-gap-analysis.md |
TechSoup summary |
docs/mappings/grantmakersIO-field-mapping.verbose.md |
Detailed Grantmakers.io analysis |
docs/mappings/grantmakersIO-field-mapping.md |
Grantmakers.io summary |
docs/mappings/CuoreeMani-field-mapping.verbose.md |
Detailed Cuoree Mani mapping |
docs/mappings/CuoreeMani-field-mapping.md |
Cuoree Mani summary |
docs/mappings/charityNavigator-field-gap-analysis.verbose.md |
Detailed Charity Navigator analysis |
docs/mappings/charityNavigator-field-gap-analysis.md |
Charity Navigator summary |
Review details
Suppressed comments (23)
docs/mappings/CuoreeMani-field-mapping.md:31
- The summary repeats the same nonexistent/uncorroborated
organization_fiscal_yeartarget. Use the existing fiscal-year-end-year base field (or mark the year as having no target) so the summary does not advertise an invalid mapping.
| 13 | Current Fiscal Year Budget | `organization_operating_budget` (pair with `organization_fiscal_year`) | organization | ✅ Close |
docs/mappings/CuoreeMani-field-mapping.verbose.md:20
organization_fiscal_yearis not referenced elsewhere as a PDC base field, while the catalog mapping ingrantmakersIO-field-mapping.md:110usesorganization_fiscal_year_end_date_year. As written, this presents a likely nonexistent target as part of an existing-field mapping.
| 13 | Current Fiscal Year Budget | `organization_operating_budget`<br>(pair with `organization_fiscal_year` to state which year) | organization | string | ✅ Close |
docs/mappings/charityNavigator-field-gap-analysis.verbose.md:21
- This source citation has the same broken relative path from
docs/mappings/; the reference should climb two directories and retain the line anchor.
From `baseFieldMap` in [`src/charityNavigator.ts:44`](src/charityNavigator.ts):
docs/mappings/charityNavigator-field-gap-analysis.verbose.md:36
- This query citation also resolves under
docs/mappings/src/and is therefore broken. Point it at the actual source file with a repository-relative path.
These fields are already requested by the `NonprofitsPublic` GraphQL query ([`src/charityNavigator.ts:74`](src/charityNavigator.ts)) but are dropped before writing to PDC.
docs/mappings/grantmakersIO-field-mapping.md:138
- These are the foundation's own charitable activities and expenses, not a changemaker's applicant proposal. Mapping them to
proposal_*fields would create a misleading proposal-shaped record; mark them as no target or specify a separate funder-program model.
| `charitable_activities[].description` | `proposal_related_programs`, `proposal_activities` |
| `charitable_activities[].expenses` | `proposal_budget` / `proposal_project_budget` |
docs/mappings/grantmakersIO-field-mapping.md:125
people[]is not a board-only list; it also supplies executive and staff mappings below. Mapping the full array to board-member names conflates those roles and would overstate board membership. Filter to director/trustee titles first.
| `people[]` | `organization_board_members_names` (file), `organization_board_members_count`, `organization_leadership` |
docs/mappings/grantmakersIO-field-mapping.md:91
- The summary repeats the same boolean/value confusion:
website_is_an_email = trueidentifies the kind of the source value; it is not the value that belongs inorganization_email. Identify and normalize the underlying email value instead.
| `website_is_an_email = true` | `organization_email` — **prevents writing a `mailto:` into the website field** |
docs/mappings/grantmakersIO-field-mapping.md:90
- This summary mapping omits the required type gate:
contact.website.cleanedcan represent an email, phone, unknown value, or URL. It should populateorganization_websiteonly when the detected type isURLand the status is accepted; otherwise route it to the appropriate field or skip it.
| `contact.website.cleaned` | `organization_website` — **use `cleaned`, not `website_verbatim`** |
docs/mappings/grantmakersIO-field-mapping.md:114
eobmf_ruling_dateis onlyyyyymmin the source, but this row presents a direct mapping to adatefield. Converting it to a full date would invent day precision; document a supported month-precision representation or keep it in a non-date field/provenance instead.
| `eobmf_ruling_date` (yyyymm) | `organization_irs_ruling_date` (date) / `organization_ruling_date` |
docs/mappings/grantmakersIO-field-mapping.md:163
INCOME_AMT/REVENUE_AMTare explicitly described as banded source values, not exact currency amounts, so they cannot be written directly to PDC currency fields;INCOME_AMTalso is not net income because no expenses are available. Leave these targets empty or define an explicit band representation instead of presenting this as a usable mapping.
| `ASSET_AMT` / `INCOME_AMT` / `REVENUE_AMT` | `organization_total_assets` / `_net_income` / `_total_revenue` (banded) |
docs/mappings/grantmakersIO-field-mapping.verbose.md:18
eobmf_ruling_dateis a date string in the referencedGrantmakersExtractedDataObj; it is not a boolean. Calling both surfaced values “derived booleans” misdescribes the available upstream data and can lead readers to ignore the ruling-date field.
- **The largest untapped org-level source is the EOBMF join the pipeline already performs.** `shared/typings/irs/all.ts` shows grantmakers.io ingests the full IRS Exempt Organizations Business Master File (`NTEE_CD`, `SUBSECTION`, `CLASSIFICATION`, `FOUNDATION`, `DEDUCTIBILITY`, `RULING`, `STATUS`, addresses, financial bands) but surfaces only **two** derived booleans from it. PDC has exact base-field targets for **~15 more** of those columns — including `organization_ntee_code`, which PDC wants and grantmakers.io already holds.
docs/mappings/grantmakersIO-field-mapping.verbose.md:207
charitable_activitiesdescribes a foundation's own direct activities, not an applicant proposal. Writing it toproposal_*/proposal_budgeton the foundation's changemaker would misrepresent the foundation as a proposal applicant; leave it unmapped or define an explicit funder-program object.
| `charitable_activities[].description` | `proposal_related_programs`, `proposal_activities` | ✅ High — 990-PF Part IX-A direct charitable activities |
| `charitable_activities[].expenses` | `proposal_budget` / `proposal_project_budget` | ⚠️ Medium |
docs/mappings/grantmakersIO-field-mapping.verbose.md:190
- The full
people[]array includes officers/key employees as well as directors/trustees; the same table uses it to derive executives and paid staff. It therefore cannot populate board-member names/count without role filtering. Restrict this mapping to board roles or mark it unavailable.
| `people[]` (full array) | `organization_board_members_names` (`file`) | ✅ High |
docs/mappings/grantmakersIO-field-mapping.verbose.md:147
contact.website.status !== 'INVALID'is not enough to validate a website: the upstream type also allowsEMAIL,PHONE-US, andUNKNOWN, andNEEDS_REVIEWis not a confirmed URL. Gateorganization_websiteon a URL type plus an accepted status, and route or skip non-URL values.
| `contact.website.status` (`VALID`/`REPAIRED`/`INVALID`/`NEEDS_REVIEW`), `.type`, `.notes[]` | _(provenance — gate ingest on `status !== 'INVALID'`)_ | ➖ |
docs/mappings/grantmakersIO-field-mapping.verbose.md:148
website_is_an_emailis only a boolean flag, not the email address itself. Mapping this row directly leaves no value to write toorganization_email; use the underlyingwebsite/contact.websitevalue when its detected type isEMAIL, normalize it to an address, and do not emit the boolean or amailto:URI.
| `website_is_an_email` = `true` | `organization_email` (the value is an address, not a URL) | ✅ High — **important**: prevents writing a `mailto:` into `organization_website` |
docs/mappings/grantmakersIO-field-mapping.verbose.md:255
- This mapping would misstate the EOBMF data: the row labels
INCOME_AMTandREVENUE_AMTas banded, whileorganization_net_incomeand the other targets are exact currency concepts. A band cannot be emitted as a currency value, and net income cannot be derived without expenses; keep these fields unmapped or model the band explicitly.
| `ASSET_AMT` / `INCOME_AMT` / `REVENUE_AMT` | `organization_total_assets` / `organization_net_income` / `organization_total_revenue` | Medium — banded |
docs/mappings/techsoup-okf-field-gap-analysis.md:84
- The verbose analysis says four mapped values are metadata in §2 and §5, but this summary calls them three. That makes the stated data-quality count internally inconsistent; it should say four (or identify a different, consistent subset).
2. **`proposal_name` names the wrong organization** ("Cumberland Gap Health Cooperative" on the Black Mountain record). Upstream content bug, propagated verbatim. Add an ingest assertion.
docs/mappings/techsoup-okf-field-gap-analysis.md:46
- The README title is an organization/display name, not evidence of a legal name, and it is already used for
organization_name. This summary should not claim a separateorganization_legal_namemapping without an explicitly labeled legal-name source.
| README title | `organization_legal_name` |
docs/mappings/techsoup-okf-field-gap-analysis.verbose.md:157
- This row has only two cells even though the table declares
Bundle data,Value, andPDC base fieldcolumns, so the target is rendered under the wrong column and the source value is missing. Split the screening label from itsall clearedvalue.
| Screening cleared: OFAC, PEP, debarment, adverse media, FATF, sanctions | `steps_to_prevent_illegal_activity` (imperfect — see §7) |
docs/mappings/techsoup-okf-field-gap-analysis.verbose.md:64
README.md#titlesupplies the bundle's display/organization name, but it does not identify a legal name; that title is already mapped toorganization_name. Treating it asorganization_legal_nameasserts an unverified legal fact.
| Legal name | synthetic-Black Mountain Workforce Partnership | `organization_legal_name` | `README.md#title` |
docs/mappings/techsoup-okf-field-gap-analysis.verbose.md:76
- These sources provide only a year (
2013), while the proposed ruling/start-date targets are date fields. Emitting a made-up month/day would lose precision; mark these as year-only or leave the date targets empty until the source provides a full date.
| Founded | 2013 | `organization_start_date` | `README.md` body |
| IRS exemption granted | 2013 ("the same year") | `organization_ruling_date` / `organization_irs_ruling_date` | `README.md` body |
docs/mappings/techsoup-okf-field-gap-analysis.verbose.md:180
- The source gives only a relative 5–8 week window, not a calendar start date. Mapping this row to
proposal_start_datewould create a false date; keep onlyproposal_durationuntil an actual start date is supplied.
| 5–8 weeks, in a between-cohort window | `proposal_duration`, `proposal_start_date` |
docs/mappings/techsoup-okf-field-gap-analysis.verbose.md:16
- This says three narrative fields, but §2 and §5.3 enumerate four:
proposal_project_outcomes,community_definition_overview,proposal_related_programs, andproposal_funding_summary. Please correct the count so the executive summary matches the detailed analysis.
- **Three genuine data-quality bugs** in the current 14 mappings should be fixed regardless of scope: `organization_status` carries a _document_ lifecycle value, `proposal_name` names the **wrong organization**, and three narrative fields carry the file's meta-disclaimer instead of its content. See §5.
- Files reviewed: 8/8 changed files
- Comments generated: 10
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
|
|
||
| | Enhancement | PDC base field | | ||
| | -------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ||
| | **`grantee_ein`** | `organization_tax_id` (recipient) ⭐⭐ **the missing join key** — LLM-derived with `confidence` + `EnhancementSource`; accept only `llm_confirmed` / `direct_match` / `high` | |
| - **`goodAsOf` from `last_updated_irs`, not `last_updated_grantmakers`** (the latter is a processing timestamp). Given the 9–18mo lag, many values arrive near or past their `valueRelevanceHours` window — a correct and useful signal, if the timestamp is honest. | ||
| - **`people[]` are named individuals with compensation.** Public on the 990-PF, but ingest at PDC's declared sensitivity (230 of 282 fields are `restricted`), not the IRS source's. | ||
| - **Consent posture differs** — this is data orgs filed under legal obligation, not chose to share. `pdc_share_data` should carry an explicit determination. | ||
| - **GivingTuesday overlap** — their repo carries a `GivTuesGrant` interface; this repo already ingests GT ([`src/getMetrics.ts`](../../src/getMetrics.ts)). Same underlying 990-PF filings, two providers → **guaranteed double-counting** unless dedup/precedence is settled first. |
|
|
||
| | Enhancement | PDC base field | Why it matters | | ||
| | -------------------------------------------------------------------------------------------------- | -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | ||
| | `grantee_ein` | `organization_tax_id` (recipient) | ⭐⭐ **The missing join key.** LLM-derived with a `confidence` of `high`/`medium`/`low` and an `EnhancementSource` (`llm`, `llm_confirmed`, `direct_name_match`, `normalized_name_match`). PDC should accept only `llm_confirmed` / `direct_match` / `high`. | |
|
|
||
| ### 6.4 Cross-reference: GivingTuesday | ||
|
|
||
| `shared/typings/grantmakers/all.ts` also defines a `GivTuesGrant` interface over the GivingTuesday 990-PF data mart (`FILEREIN`, `SIGOCPYRBNBN1`, `SIGOCPYAMOUN`, `SIGOCPYPOGOC`, …). Since this repo already ingests GivingTuesday ([`src/getMetrics.ts`](src/getMetrics.ts), branch `ENH-296`), **the same underlying 990-PF grants are reachable from two providers.** Deduplication strategy and provider precedence should be settled before adding grantmakers.io grants, or PDC will double-count. |
| ## Gotchas / data-quality bugs | ||
|
|
||
| 1. **`organization_status` = `"stable"` is the document lifecycle value**, not an org status — and PDC marks that field `public`. The real status is `ELIGIBLE` in `verification.md` → belongs in `review_status`. | ||
| 2. **`proposal_name` names the wrong organization** ("Cumberland Gap Health Cooperative" on the Black Mountain record). Upstream content bug, propagated verbatim. Add an ingest assertion. |
| | Sustain/expand community visits; recruitment; general operating support | `proposal_activities`, `proposal_funding_category` | | ||
| | "Neither of us could run our half without the other" | `proposal_related_programs`, `significant_other_funders` | | ||
|
|
||
| ⚠️ **This file's body is the wrong organization's content** — it describes a clinic with a nurse practitioner and a truck, and its `title` names "synthetic-Cumberland Gap Health Cooperative". That title is what currently populates `proposal_name`. See §5. |
|
|
||
| 3. **Four values are metadata, not content.** `proposal_project_outcomes`, `community_definition_overview`, `proposal_related_programs`, and `proposal_funding_summary` all carry the file's authorial note ("Fabricated.", "Illustrative only; not part of the published civic/0.6 profile.") rather than the file's substance. | ||
|
|
||
| 4. **Everything is typed `"string"`.** PDC declares 7 `currency`, 5 `email`, 4 `date`, 1 `number`, 1 `phone_number`, and 8 `file` fields. If financials and dates are added per §3, `organization_total_revenue` / `organization_total_assets` must be emitted as `currency`, `organization_irs_ruling_date` as `date`, and `organization_volunteers` as `number`. |
| | **Total achievable with no schema change** | **~69** | **24.5%** | | ||
| | Requires new base fields (§7) | ~11 concepts | — | | ||
|
|
||
| By PDC category, the achievable additions land as: `organization` +22, `project` +20, `evaluation` +9, `outcomes` +3, `budget` +4, `methodology` +5, `partnerships` +1, `sustainability` +1. |
|
|
||
| ## 9. Recommended order of work | ||
|
|
||
| 1. **Fix the six data-quality defects in §5** — smallest effort, highest correctness return. In particular stop mapping the document `status` to `organization_status`, and add `goodAsOf` from `generated.at` / `sources[].last_modified` / `verified.at`. |
|
I am still not sure these docs belong here, but the worst that can happen is that they, like all other documentation, get stale. Copilot had some suggestions (more than I expected)! |
There was a problem hiding this comment.
🟡 Changes recommended
Broken references and unresolved mapping, schema, privacy, and data-quality inconsistencies remain.
Get a fresh assessment by requesting another Copilot review.
Review details
Suppressed comments (33)
docs/mappings/CuoreeMani-field-mapping.md:31
- The summary repeats the undefined
organization_fiscal_yearcode. Use the existingorganization_fiscal_year_end_date_yearfield name, or omit the pairing until the target catalog confirms a different field.
| 13 | Current Fiscal Year Budget | `organization_operating_budget` (pair with `organization_fiscal_year`) | organization | ✅ Close |
docs/mappings/CuoreeMani-field-mapping.verbose.md:20
organization_fiscal_yeardoes not appear elsewhere in the repository's PDC field mappings; the existing field name for a year isorganization_fiscal_year_end_date_year(used in the Grantmakers mapping). Recommending the former leaves consumers looking for a base field that is not defined here.
| 13 | Current Fiscal Year Budget | `organization_operating_budget`<br>(pair with `organization_fiscal_year` to state which year) | organization | string | ✅ Close |
docs/mappings/charityNavigator-field-gap-analysis.verbose.md:3
- This relative link is resolved from
docs/mappings, sosrc/charityNavigator.tspoints atdocs/mappings/src/charityNavigator.ts, not the repository'ssrc/charityNavigator.ts. Use the repository-relative../../src/charityNavigator.tspath.
**Subject:** [`src/charityNavigator.ts`](src/charityNavigator.ts) field mapping (`baseFieldMap`)
docs/mappings/charityNavigator-field-gap-analysis.verbose.md:21
- This source link has the same broken base path: from
docs/mappings,src/charityNavigator.ts:44resolves underdocs/mappings/src. Point it to the actual source file (ideally with a line anchor).
From `baseFieldMap` in [`src/charityNavigator.ts:44`](src/charityNavigator.ts):
docs/mappings/charityNavigator-field-gap-analysis.verbose.md:36
- This link also resolves to a nonexistent
docs/mappings/srcdirectory. The query is in the repository root'ssrcdirectory, so the link needs../../src/charityNavigator.ts#L74.
These fields are already requested by the `NonprofitsPublic` GraphQL query ([`src/charityNavigator.ts:74`](src/charityNavigator.ts)) but are dropped before writing to PDC.
docs/mappings/chicago_partner_org-field-mapping.md:122
- The summary has the same problem:
school_years.numberis not itself a date suitable forgoodAsOf. Specify the date derivation or preserve the school-year period only as provenance.
- **`goodAsOf` from the school year, not `updated_at`.** A 2019 profile edited in 2026 is still a 2019 fact. Use `school_years.number`; keep `updated_at` as batch metadata.
docs/mappings/chicago_partner_org-field-mapping.md:42
- This summary lists
employees_countas a coreorganization_paid_staffmapping even though the Gotchas section says it is a portal-login count and should otherwise be left empty. Remove it from the reachable mapping (and coverage totals) or clearly mark it as excluded to avoid publishing false staff counts.
| `employees_count` | `organization_paid_staff` | ⚠️ **counts portal logins, not staff** |
docs/mappings/chicago_partner_org-field-mapping.verbose.md:228
- A school-year label such as
2024–25is a period, not a point-in-time date. DerivinggoodAsOfdirectly fromschool_years.numbercan create a fabricated freshness date; use an explicit period start/end date if the schema provides one, otherwise keep the school year in provenance and do not synthesizegoodAsOf.
6. **`goodAsOf` must come from the school year, not `updated_at`.** `updated_at` records when a row was last touched in the application; `school_year_id` records what period the value describes. A 2019 profile edited in 2026 is still a 2019 fact. Resolve `school_year_id` → `chicago.school_years.number` and derive `goodAsOf` from that, using `updated_at` only as batch metadata.
docs/mappings/chicago_partner_org-field-mapping.verbose.md:270
- §3.2 presents
district_funding_centsas mappable tosignificant_other_funders, but this section calls the same value "no PDC target". Distinguish an exact target from the lossy interim fallback so an implementer does not treat a single funding stream as a general partnerships fact.
| **District funding received** (`district_funding_cents`) | Revenue from one named public source. `organization_total_revenue` would misstate it as a total | `organization_funding_by_source`, or a repeatable funder/amount pair |
docs/mappings/chicago_partner_org-field-mapping.verbose.md:3
- The only schema source is identified by a contributor's workstation-local path, so readers and CI cannot follow or reproduce this analysis from a checkout. Replace it with a repository-relative or stable external reference, or include the workbook as a reviewable artifact.
**Subject:** `C:\Data\Parliament\chicago_partner_org_metadata.xlsx` — the `columns` tab (schema inventory) and the `relationships` tab (join contract)
docs/mappings/chicago_partner_org-field-mapping.verbose.md:326
- Filtering
details.state = 'approved'here discards every archived school-year row, even though the mapping earlier treatsdetails.stateas the source fororganization_status(archived= inactive) and recommends one value per school year. Either retain both states for historical status values or explicitly scope the feed to active years and remove the archived-status expectation.
5. **Apply the §8 gates in the extract query**, not downstream — `restrict_access`, `mask_pii`, `details.state = 'approved'`, `allocations.state` approved, and `*_type = 'Organization'` on every polymorphic join.
docs/mappings/chicago_partner_org-field-mapping.verbose.md:324
- The source description says the EIN is nullable and recommends measuring its fill rate, but this step unconditionally proposes one changemaker per organization keyed on EIN. Organizations without an EIN need a stable artlook
idfallback or an explicit exclusion rule; otherwise this implementation plan drops part of the source.
3. **Ship the organization core first** — §3.1 identity + §3.2 profile + §3.3 address. Roughly 25 fields, high confidence, one changemaker per `chicago.organizations` row keyed on EIN.
docs/mappings/chicago_partner_org-field-mapping.verbose.md:93
- This is listed as a mapping to
organization_paid_staff, but §5.2 says the value counts portal accounts rather than staff and recommends leaving the field empty. Keeping it in the organization mapping can still lead to false headcount data; mark it as excluded rather than reachable.
| `employees_count` | `organization_paid_staff` | string | ⚠️ **Low** | Counts _portal users affiliated with the org_, not headcount — see §5 |
docs/mappings/grantmakersIO-field-mapping.md:60
- The summary still recommends mapping the filing year directly to
proposal_date, which would produce an invalid or misleading date value. Treattax_yearas provenance only and derivegoodAsOffromlast_updated_irs.
| `tax_year` | `proposal_date` + drives `goodAsOf` | ⚠️ filing year ≠ award date |
docs/mappings/grantmakersIO-field-mapping.md:125
- The summary repeats the same over-broad mapping:
people[]is also used to derive executives, so its full length cannot be treated as board membership. Use only records identified as directors/trustees for the board fields.
| `people[]` | `organization_board_members_names` (file), `organization_board_members_count`, `organization_leadership` |
docs/mappings/grantmakersIO-field-mapping.md:138
- The summary presents ongoing foundation activities as proposal fields without the scope caveat in the verbose analysis. This can cause consumers to interpret Part IX-A activities as grant applications or funded projects; label it as an explicitly lossy fallback.
| `charitable_activities[].description` | `proposal_related_programs`, `proposal_activities` |
| `charitable_activities[].expenses` | `proposal_budget` / `proposal_project_budget` |
docs/mappings/grantmakersIO-field-mapping.md:171
- The summary uses
direct_match, but the source enum listed in the verbose mapping isdirect_name_match. This token mismatch makes the proposed acceptance gate ambiguous or impossible to implement.
| **`grantee_ein`** | `organization_tax_id` (recipient) ⭐⭐ **the missing join key** — LLM-derived with `confidence` + `EnhancementSource`; accept only `llm_confirmed` / `direct_match` / `high` |
docs/mappings/grantmakersIO-field-mapping.md:192
- This link points to
getMetrics.ts, which only reports endpoint counts; it is not the GivingTuesday importer. Link tosrc/givingTuesday.tsinstead, otherwise readers are sent to unrelated code when following the deduplication warning.
- **GivingTuesday overlap** — their repo carries a `GivTuesGrant` interface; this repo already ingests GT ([`src/getMetrics.ts`](../../src/getMetrics.ts)). Same underlying 990-PF filings, two providers → **guaranteed double-counting** unless dedup/precedence is settled first.
docs/mappings/grantmakersIO-field-mapping.md:125
- The summary omits the required conversion for
organization_board_members_names:people[]is structured data and the target is afilefield, not a directly writable string/array. Add the serialization/upload requirement so this does not get implemented as a direct mapping.
| `people[]` | `organization_board_members_names` (file), `organization_board_members_count`, `organization_leadership` |
docs/mappings/grantmakersIO-field-mapping.verbose.md:85
tax_yearis only a filing year, not a grant/proposal date, and the target is a date field. Writing a bare number such as2023toproposal_datewould either be invalid or falsely imply an award date. Keep the year as provenance and use the IRS source timestamp forgoodAsOfunless an actual award date is available.
| `tax_year` | number | `proposal_date`; also drives `goodAsOf` | project | ⚠️ Filing year, not award date |
docs/mappings/grantmakersIO-field-mapping.verbose.md:191
- The full
people[]array also supplies the executive-administrator rows immediately below, so it is not a board-only collection. Mapping the whole array and its length to board names/count will include officers and other listed people; filter to director/trustee roles before populating these board fields.
| `people[]` (full array) | `organization_board_members_names` (`file`) | ✅ High |
| `people[].length` | `organization_board_members_count` | ✅ High |
docs/mappings/grantmakersIO-field-mapping.verbose.md:195
people[].compensationincludes the same mixed population used for executive extraction, so it is not automatically governing-body compensation. Without filtering board/trustee roles, this mapping can publish officer or staff pay as board compensation.
| `people[].compensation` (+ `hours`) | `organization_governing_body_compensation` | ✅ High — direct match, and rarely available elsewhere |
docs/mappings/grantmakersIO-field-mapping.verbose.md:267
direct_matchis not one of theEnhancementSourcevalues enumerated in this same row (direct_name_matchis). As written, an implementer could gate out the actual direct-name matches; use the exact enum name and keep the source/confidence checks distinct.
| `grantee_ein` | `organization_tax_id` (recipient) | ⭐⭐ **The missing join key.** LLM-derived with a `confidence` of `high`/`medium`/`low` and an `EnhancementSource` (`llm`, `llm_confirmed`, `direct_name_match`, `normalized_name_match`). PDC should accept only `llm_confirmed` / `direct_match` / `high`. |
docs/mappings/grantmakersIO-field-mapping.verbose.md:207
- These are the foundation's ongoing Part IX-A charitable activities, not proposal submissions. Writing them to
proposal_*fields with high confidence makes them look like funded proposals; mark this as a lossy fallback (or keep it organization-scoped) and lower the confidence.
| `charitable_activities[].description` | `proposal_related_programs`, `proposal_activities` | ✅ High — 990-PF Part IX-A direct charitable activities |
| `charitable_activities[].expenses` | `proposal_budget` / `proposal_project_budget` | ⚠️ Medium |
docs/mappings/grantmakersIO-field-mapping.verbose.md:294
- This
getMetrics.tslink is resolved relative todocs/mappings, so it points atdocs/mappings/src/getMetrics.ts, which does not exist. Use the repository-relative../../src/getMetrics.tspath.
`shared/typings/grantmakers/all.ts` also defines a `GivTuesGrant` interface over the GivingTuesday 990-PF data mart (`FILEREIN`, `SIGOCPYRBNBN1`, `SIGOCPYAMOUN`, `SIGOCPYPOGOC`, …). Since this repo already ingests GivingTuesday ([`src/getMetrics.ts`](src/getMetrics.ts), branch `ENH-296`), **the same underlying 990-PF grants are reachable from two providers.** Deduplication strategy and provider precedence should be settled before adding grantmakers.io grants, or PDC will double-count.
docs/mappings/grantmakersIO-field-mapping.verbose.md:196
people[].hours > 0does not identify employees or paid staff: the available person records include board/officer people, and the typings expose hours and compensation but no staff classification. Mapping that count toorganization_paid_staffcan materially overstate staffing; leave the field unmapped or document a separately validated proxy.
| `people[]` where `hours > 0` (count) | `organization_paid_staff` | ⚠️ Medium — better than the boolean below |
docs/mappings/grantmakersIO-field-mapping.verbose.md:190
people[]is a structured array, whileorganization_board_members_namesis documented here as a PDCfilefield. The array cannot be written directly as that field value; document the required file serialization/upload or use a compatible text representation before calling this a high-confidence direct mapping.
| `people[]` (full array) | `organization_board_members_names` (`file`) | ✅ High |
docs/mappings/techsoup-okf-field-gap-analysis.md:63
- This row presents the funding file as mappable, but the file is explicitly titled and written for Cumberland Gap rather than Black Mountain. An implementer following this summary could attach the clinic's funding request to the Black Mountain record; mark it blocked until the source is reassigned or corrected.
| `what_i_need_funding_for.md` body | `proposal_funding_summary`, `proposal_need`, `proposal_purpose_statement`, `proposal_activities`, `proposal_funding_category` |
docs/mappings/techsoup-okf-field-gap-analysis.verbose.md:257
- These category counts sum to 65 (22 + 20 + 9 + 3 + 4 + 5 + 1 + 1), but §8 says there are only ~55 additional fields and ~69 total. Since each base field belongs to one category, reconcile this breakdown with the coverage table before using it for prioritization.
By PDC category, the achievable additions land as: `organization` +22, `project` +20, `evaluation` +9, `outcomes` +3, `budget` +4, `methodology` +5, `partnerships` +1, `sustainability` +1.
docs/mappings/techsoup-okf-field-gap-analysis.verbose.md:263
- §5 contains seven numbered data-quality issues, so this work item is off by one and makes the recommended order internally inconsistent. Either say seven or avoid hard-coding the count.
1. **Fix the six data-quality defects in §5** — smallest effort, highest correctness return. In particular stop mapping the document `status` to `organization_status`, and add `goodAsOf` from `generated.at` / `sources[].last_modified` / `verified.at`.
docs/mappings/techsoup-okf-field-gap-analysis.verbose.md:16
- This bullet says three narrative fields carry the metadata disclaimer, but §5.3 identifies four:
proposal_project_outcomes,community_definition_overview,proposal_related_programs, andproposal_funding_summary. Please make the count consistent with the detailed list.
- **Three genuine data-quality bugs** in the current 14 mappings should be fixed regardless of scope: `organization_status` carries a _document_ lifecycle value, `proposal_name` names the **wrong organization**, and three narrative fields carry the file's meta-disclaimer instead of its content. See §5.
docs/mappings/techsoup-okf-field-gap-analysis.verbose.md:157
- §7 correctly says the screening battery has no existing PDC target because
steps_to_prevent_illegal_activitydescribes prevention methodology, not screening results. Listing it here as an evaluation mapping both suggests a semantically wrong field and inflates the existing-field coverage; remove it from the mappable table or explicitly mark it as a lossy interim workaround and recalculate the totals.
| Screening cleared: OFAC, PEP, debarment, adverse media, FATF, sanctions | `steps_to_prevent_illegal_activity` (imperfect — see §7) |
docs/mappings/techsoup-okf-field-gap-analysis.verbose.md:224
- §5.2 says this file's title and body are actually the Cumberland Gap clinic's content. Emitting it as a proposal against the Black Mountain changemaker would still publish another organization's funding request on the wrong record; it must be reassigned or excluded before the two-proposal recommendation is applied.
**Recommendation:** emit these as **two separate proposals** against the same changemaker before expanding proposal-side coverage. If the target schema cannot express that yet, map only the funding need to `proposal_*` fields and keep the volunteer request in organization-scoped fields (`organization_workplace_issues`) until it can.
- Files reviewed: 10/10 changed files
- Comments generated: 11
- Review effort level: Lite
| ## Phase 1 — add now (target exists, currently empty) | ||
|
|
||
| Requires adding these to the GraphQL query + interface + `baseFieldMap`. No PDC schema change. |
| Charity Navigator's public nonprofit type exposes more than the 12 fields the current query selects. Per Charity Navigator's own [`cn-examples`](https://github.com/CharityNavigator/cn-examples) query samples, the public nonprofit object also includes a **structured address** and organization URLs. These have **ready, currently-empty PDC targets**: | ||
|
|
||
| | CN attribute (available, not queried) | Recommended PDC base field | PDC data type | Notes | |
| | | | | ||
| | ---------------------- | ----------------------------------------------------------------------------------------- | | ||
| | **Source** | artlook Chicago — arts-education partner portal (Rails/Postgres, multi-tenant) | | ||
| | **Schema doc** | `C:\Data\Parliament\chicago_partner_org_metadata.xlsx` — `columns` + `relationships` tabs | |
|
|
||
| ## 🚩 Read first | ||
|
|
||
| 1. **`chicago.*` is a strict subset of `public.*`.** `public` is the multi-tenancy _template_, not Chicago data. Read entities from `chicago.*`, shared lookups (`users`, `positions`, `grades`, `program_types`, `communities`) from `public.*`. **Ingesting both duplicates every changemaker.** |
| # Chicago Partner Organizations (artlook) → PDC Field Mapping | ||
|
|
||
| **Subject:** `C:\Data\Parliament\chicago_partner_org_metadata.xlsx` — the `columns` tab (schema inventory) and the `relationships` tab (join contract) | ||
| **Supporting context:** the companion `chicago_partner_org_metadata.md` in the same folder, which adds entity groupings, the allocation matrix, and `state`-field semantics that the spreadsheet alone does not carry |
|
|
||
| ### 2.1 The two schemas are structurally identical | ||
|
|
||
| A column-level diff confirms `chicago.*` is a **strict subset** of `public.*`: every shared table has identical columns, and `public` additionally carries six lookup/aux tables (`demographics`, `engagement_types`, `integration_types`, `schedule_types`, `sign_ins`, `space_types`). |
| | All programs for an org, rendered | `proposal_related_programs` | ⭐ **High** | | ||
| | `programs.description` | `proposal_activities`, `proposal_summary` / `proposal_abstract` | ✅ High | | ||
| | `programs.discipline_id` → `chicago.disciplines.name` | `proposal_program_area`, `proposal_focus` | ✅ High | | ||
| | `programs.school_year_id` | `proposal_duration` / → `goodAsOf` | ⚠️ Low — a year label, not a duration | |
|
|
||
| ### EOBMF join — ~15 PDC fields, already ingested | ||
|
|
||
| They join the full IRS Business Master File but surface only two derived booleans from it. |
| - **Both datasets map well, but asymmetrically.** The **Profiles** dataset (163,004 foundations, 63 fields) maps cleanly onto **~40 existing PDC base fields** — it is essentially a pre-parsed 990-PF organization record and PDC already has targets for nearly all of it. The **Grantees** dataset (4,882,146 grant records, 21 fields) maps to only **~12** — not because the data is poor, but because **PDC has no grant/award object**, and because of the identity problem below. | ||
| - **⚠️ The single biggest blocker is that grantee records carry no EIN.** IRS Form 990-PF Part XV does not require the filer to supply a recipient EIN (unlike Form 990 Schedule I). Grantee identity is a free-text name the funder typed. Without an EIN there is no reliable join to a PDC changemaker, and 4.9M name strings cannot be safely resolved into organization records. | ||
| - **Grantmakers.io is already solving that**, and it is the highest-value thing to ask them for. Their typings define an `Enhancements` layer with an LLM-derived **`grantee_ein`** carrying a confidence grade, plus `grantee_classification`, `grantee_mission`, and a `privacy_shield` flag for individual recipients. None of it is in the public index today. **If PDC can obtain the enhanced grants collection, the Grantees mapping goes from ~12 fields to a genuine changemaker-plus-funding-history feed.** | ||
| - **The largest untapped org-level source is the EOBMF join the pipeline already performs.** `shared/typings/irs/all.ts` shows grantmakers.io ingests the full IRS Exempt Organizations Business Master File (`NTEE_CD`, `SUBSECTION`, `CLASSIFICATION`, `FOUNDATION`, `DEDUCTIBILITY`, `RULING`, `STATUS`, addresses, financial bands) but surfaces only **two** derived booleans from it. PDC has exact base-field targets for **~15 more** of those columns — including `organization_ntee_code`, which PDC wants and grantmakers.io already holds. |
| | Title: "Three funder reports from one participant record…" | `proposal_project_title` | | ||
| | "The need, in the org's words" | `proposal_need`, `proposal_context` | | ||
| | "What a volunteer would do" (7 numbered steps) | `proposal_activities`, `proposal_tactics_and_methods` | | ||
| | 5–8 weeks, in a between-cohort window | `proposal_duration`, `proposal_start_date` | |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
bickelj-agent
left a comment
There was a problem hiding this comment.
Reviewed all 10 new mapping docs against the live PDC base-field catalog; the field names and types check out. Copilot thread status in the current documents: 0 resolved, 21 unresolved (3 partially addressed, 18 untouched). Three new issues are in-line. — DeepSeek-V4.1-Flash
| | `zip` | `organization_postal_code` | string | ✅ High (`NOT NULL`) | | ||
| | `latitude` | `organization_latitude_epsg_4326` | string | ✅ High | | ||
| | `longitude` | `organization_longitude_epsg_4326` | string | ✅ High | | ||
| | `community_area` | `organization_neighborhood` | string | ⭐ **High** | Chicago's 77 community areas — a genuinely good fit for a field PDC rarely gets filled | |
There was a problem hiding this comment.
This row has a fifth cell, but the §3.3 table header has four columns. The table renders wrong and this file fails CI: npx prettier --check docs/mappings warns on this file, and the prettier job runs npm run lint:prettier. Give the table a 'Notes' column or drop the trailing cell, then run npm run format:prettier. — DeepSeek-V4.1-Flash
|
|
||
| 1. **`organization_status` = `"stable"` is the wrong `status`.** The bundle's frontmatter `status:` is an OKF _document lifecycle_ value (`stable` / `draft` — `technology/capability.md` uses `draft`). PDC's `organization_status` is a **public** field describing the organization. Mapping one to the other publishes a document state as an organizational fact. The organization's actual status lives in `verification.md` ("Current status: ELIGIBLE") and belongs in `review_status`. | ||
|
|
||
| 2. **`proposal_name` names the wrong organization** — "synthetic-Cumberland Gap Health Cooperative — What I need funding for", on the Black Mountain record. This is an upstream content bug in `what_i_need_funding_for.md`, but the exporter propagates it verbatim. Worth an ingest-time assertion that a proposal title's organization prefix matches the changemaker. |
There was a problem hiding this comment.
This claim is inverted. In the live what_i_need_funding_for.md, the frontmatter title is synthetic-Black Mountain Workforce Partnership — What I need funding for. So proposal_name holds the correct organization. Only the body carries the clinic text. Document it as a body/title mismatch, not a wrong-org proposal_name. The same error is at line 169 and in the summary (line 84). — DeepSeek-V4.1-Flash
| - **Yes — substantially more.** The feed maps **14 of 282** base fields. Another **~55 existing base fields** can be populated from the bundle as it stands today, with no new PDC fields required. That is roughly a **4× increase in coverage** from the same source files. | ||
| - **The root cause is mechanical, not semantic.** The exporter reads only YAML **frontmatter** (`title`, `description`, and four `x-civic` keys). It never reads the markdown **body** — and the body is where every number, the mission statement, the address, the financials, the program list, and the verification determination live. Fixing the extraction layer, not the mapping vocabulary, is where the leverage is. | ||
| - **Four highest-value additions, all trivially available:** `organization_website` (already sits in frontmatter as `resource:` and is simply dropped), `organization_mission_statement`, the address quintet (`organization_street_address_1` / `city` / `state_province` / `postal_code` / `county`), and the financial trio (`organization_total_revenue`, `organization_total_assets`, `organization_paid_staff`). | ||
| - **Three genuine data-quality bugs** in the current 14 mappings should be fixed regardless of scope: `organization_status` carries a _document_ lifecycle value, `proposal_name` names the **wrong organization**, and three narrative fields carry the file's meta-disclaimer instead of its content. See §5. |
There was a problem hiding this comment.
three narrative fields contradicts §5 item 3 and the summary gotcha, which both say four values. This bullet also counts three bugs, while §5 lists seven and §9 says six. Use one count in all three places. — DeepSeek-V4.1-Flash
There are normal and 'verbose' versions here. Verbose is meant for ai ingestion