Skip to content

Added AllowFieldCountMismatch property to CsvConfiguration - #985

Draft
michelebastione wants to merge 912 commits into
mini-software:masterfrom
michelebastione:csv-jagged-rows
Draft

Added AllowFieldCountMismatch property to CsvConfiguration#985
michelebastione wants to merge 912 commits into
mini-software:masterfrom
michelebastione:csv-jagged-rows

Conversation

@michelebastione

@michelebastione michelebastione commented Jul 19, 2026

Copy link
Copy Markdown
Collaborator

The implementation of this property makes jagged rows being returned as they are instead of having an exception being thrown when a mismatched number of fields is found.

Resolves #979

Summary by CodeRabbit

  • New Features

    • Added an option to interpret empty CSV fields as default values.
    • Added support for reading rows with varying numbers of fields when field-count mismatches are allowed.
    • Improved handling of missing and additional columns during CSV import.
  • Bug Fixes

    • Corrected reported column names for missing-column errors.
  • Deprecation

    • Replaced the previous empty-string handling option with the new configuration setting.

jiaguangli and others added 30 commits July 27, 2024 08:48
When tabSelected="1", the resulting Excel will say "Cannot do this for multiple selected areas"

Signed-off-by: Gary Jia <35099424+jiaguangli@users.noreply.github.com>
…tware#647)

* Fixed an issue with QueryAsDataTable that only header rows cannot be read correctly

* handling null value
…mplify code (mini-software#651)

* feat: 增加 MiniExcelDataReaderBase 基类

* refactor: 使用基类精简代码

* Update MiniExcelDataReaderBase.cs

Signed-off-by: Wei Lin <shps951002@gmail.com>

---------

Signed-off-by: Wei Lin <shps951002@gmail.com>
Co-authored-by: Wei Lin <shps951002@gmail.com>
* Add tests for fast mode

* Fix dimension writing in FastMode
mini-software#679)

* Formula attribute added to support in rows with dto or dynamic attributes

* Change Formula attribute to ColumnType (Value, Formula)

---------

Co-authored-by: Eulises Vargas <eulises.vargas@iconstruye.onmicrosoft.com>
* Add async implementation of frozen rows and columns

* Add unit tests for async implementation of frozen rows and columns

* Fix formatting and delete temp file in unit test

* move freezing top row implementation to DefaultOpenXml.cs
…ut the value as text. Preventing of losing leading zeroes
* Add AutoWidth functionality for async write

* Add MiniExcelAutoAdjustWidthTests

* Implement autowidth for sync write
…ormatting Delegate (mini-software#700)

* Add DynamicColumnFirst in config;

* Handle DynamicColumnFirst value assign;
Modify DynamicColumns ignore case;

* Handle CustomFormatter logic;
…e#706)

* feat(MiniExcelDataReaderBase): Add asynchronous support

* fix `ReadAsync` bug

* remove the error submission code

* add unit test

* optimization cancellation

* optimization disposeAsync

* optimized code

* chore: bump version 1.36.1

* refactor: 格式化代码

---------

Co-Authored-By: Argo Zhang <argo@live.ca>
* feat: support insert sheet

* optimize code

* fix: async method bug
…iting IAsyncEnumerable (mini-software#712)

* Generalize excel writing with a common write adapter

* Remove testcase for single row in empty table

* Implement sync IExcelWriteAdapter

* Remove testcase for single row in empty table

* Add back support for writing IMiniExcelDataReader

* Add support for writing AsyncEnumerable

* Add cancellation support

* Add MiniExcelWriteAdapterFactory

* Implement write adapters for csv writing

* cancel break change

* fix build bug

* fix get props bug

* cancel some adjust

* clean code

---------

Co-authored-by: izanhzh <hzh990094740@outlook.com>
…rful (mini-software#715)

* feat(DynamicExcelColumn): make the `CustomFormatter` property more powerful

* support `IEnumerable` data
* feat(ExcelNumberFormat): extend the SectionType

* cancel the missubmit code

* Update SheetStyleBuildContext.cs

Signed-off-by: Wei Lin <shps951002@gmail.com>

---------

Signed-off-by: Wei Lin <shps951002@gmail.com>
Co-authored-by: Wei Lin <shps951002@gmail.com>
…atter` execution (mini-software#722)

* adjust the location of the `CustomFormatter` execution

* fix bug
Signed-off-by: Michele Bastione <michele.bastione@gmail.com>
* Minor xml generation refactoring

* Changed sheets and files' ids to be deterministic

Generated Excel files are now always consistent when the data source is the same, save for the metadata of the zip entries.
@coderabbitai

coderabbitai Bot commented Jul 19, 2026

Copy link
Copy Markdown

Review Change Stack

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 0e59f1dc-abef-458a-9f66-7d72740a5f27

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

CsvReader.QueryAsync now handles empty fields and variable-width rows through the new ReadEmptyFieldsAsDefault option and existing mismatch configuration. Tests cover headered, headerless, and mapped jagged CSV rows.

Changes

CSV jagged-row handling

Layer / File(s) Summary
Empty-field configuration contract
src/MiniExcel.Csv/CsvConfiguration.cs
Adds ReadEmptyFieldsAsDefault and makes obsolete ReadEmptyStringAsNull delegate to it.
Row parsing and result mapping
src/MiniExcel.Csv/CsvReader.cs
Refactors row parsing, mismatch validation, synthetic columns, filler fields, and empty-field conversion for headered and headerless CSV data.
Jagged-row and exception coverage
tests/MiniExcel.Csv.Tests/Main/MiniExcelCsvAsyncTests.cs, tests/MiniExcel.Csv.Tests/Main/MiniExcelCsvTests.cs, tests/MiniExcel.Csv.Tests/Main/Models.cs
Adds jagged-row tests and DTO fields, and updates column-name and dictionary assertions.

Estimated code review effort: 3 (Moderate) | ~20 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly names the main change: adding CsvConfiguration support for mismatched CSV field counts.
Linked Issues check ✅ Passed The changes add AllowFieldCountMismatch, update CSV parsing to tolerate jagged rows, and add tests matching issue #979.
Out of Scope Changes check ✅ Passed The extra CSV empty-field and test/model updates appear to support the same jagged-row parsing feature and are not unrelated.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/MiniExcel.Csv/CsvReader.cs`:
- Around line 67-133: Update the invalid-row check before the header and
headerless branches to reject any field-count mismatch when
AllowFieldCountMismatch is false, not only rows where fields.Length is less than
headerRow.Count. Preserve the existing ColumnNotFoundException construction and
ensure both hasHeaderRow paths exit through this check before assigning fields.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f567c7b8-2687-4a1f-bf0f-0c366b9d374b

📥 Commits

Reviewing files that changed from the base of the PR and between 3266e8b and 87445aa.

📒 Files selected for processing (5)
  • src/MiniExcel.Csv/CsvConfiguration.cs
  • src/MiniExcel.Csv/CsvReader.cs
  • tests/MiniExcel.Csv.Tests/Main/MiniExcelCsvAsyncTests.cs
  • tests/MiniExcel.Csv.Tests/Main/MiniExcelCsvTests.cs
  • tests/MiniExcel.Csv.Tests/Main/Models.cs

Comment thread src/MiniExcel.Csv/CsvReader.cs
@michelebastione
michelebastione marked this pull request as draft July 19, 2026 16:01
… and benchmarks projects (mini-software#986)

* Added documentation to the MiniExcelConverter utility class

* Fixed warnings in tests and benchamrks projects

* Minor adjustments
…tware#988)

* Removed unused System.Security.Cryptography.Xml package

* Removed some tests of questionable usefulness

* Refactored benchmarks sample data

* Updated .gitattribtues to treat all tests data files as binary data

* Fixed typo
@michelebastione
michelebastione force-pushed the csv-jagged-rows branch 2 times, most recently from 077f1e6 to 6d1fa3e Compare July 22, 2026 22:00
…and stats

Signed-off-by: Michele Bastione <michele.bastione@gmail.com>
michelebastione and others added 15 commits July 23, 2026 23:50
* Moved FluentMapping logic to its own MiniExcel.OpenXml.FluentMapping assembly

This makes it easier to mantain and also makes the main MiniExcel.OpenXml assembly smaller for the users who don't care for the fluent mapping

* Updated System.Security.Cryptography.Xml package

Manually added latest version of the package where NPOI brought in a vulnerable one

* Minor fixes
* Added CODE_OF_CONDUCT.md and CONTRIBUTING.md files

* Minor fixes
…penXml) (mini-software#993)

* Fix template calcChain regeneration and $= formula cells (v2 / MiniExcel.OpenXml)

Porting the v1.x template-render fixes to the v2 OpenXml line, where the
engine moved to src/MiniExcel.OpenXml/ and a LINQ-to-XML rewrite.

Two defects made Excel reject an otherwise-valid rendered package:

- $= formula cells kept their source t="inlineStr" attribute after the
  inline string was replaced with an <f> child — a cell typed inlineStr
  with no <is> is schema-invalid. The attribute is now removed.

- The regenerated calcChain entry took the cell address from the running
  child-index, which is wrong for sparse rows (cells with no content are
  not emitted, so position != column). The address is now read from the
  cell's own r attribute.

- The template's own calcChain was reused/copied even though row insertion
  shifts every formula cell, leaving stale entries; and an empty calcChain
  (no rendered formulas) was written, which is schema-invalid. The chain is
  now regenerated only when formulas were rendered and dropped entirely
  otherwise — Excel rebuilds it on open.

The <f> namespace fix from the v1 change is not needed here: v2's
XElement-based serialization already emits it in the spreadsheetml namespace.

* Removed obsolete FluentMapping folder from project

* Converted usage of some raw strings to typed constants

* Changed the ProcessFormulas method to calculate cell references based on the full "r" attribute when available

* Added regression tests for new handling of calcChain data

* Minor fix in the generation of the path/name map for the sheets

* Removed ghost calcChain references across the generated package

Added logic to remove calcChain references from [Content_Types].xml and workbook.xml.rels when the calcChain is not to be copied from the template's archive.

* Fixed minor issue with the extrapolation of calcChain references

---------

Co-authored-by: Michele Bastione <michele.bastione@gmail.com>
- Add error handling with custom error types in `error.rs`.
- Create core library structure in `lib.rs` with modules for cell handling, error management, options, reading, writing, and serialization helpers.
- Define `ReadOptions` and `WriteOptions` in `options.rs` for configuring read/write behavior.
- Implement `XlsxReader` in `reader.rs` for reading XLSX files, including dynamic row handling and deserialization.
- Add serialization helpers in `serde_helpers.rs` for date, time, and duration types.
- Implement `XlsxWriter` in `writer.rs` for writing XLSX files, including schema validation and cell formatting.
- Create common test utilities in `common/mod.rs` for file path handling.
- Add tests for reading dynamic and typed rows, handling empty rows, and round-trip writing/reading of XLSX files.
- Set Rust toolchain version and formatting options in `rust-toolchain.toml` and `rustfmt.toml`.
- Introduced a new `MiniExcel` struct as a convenience entry point for querying XLSX files.
- Implemented `StreamingRows` and `StreamingTypedRows` for efficient row streaming and deserialization.
- Updated tests to utilize the new `MiniExcel` API for querying and validating data from XLSX files.
- Enhanced error handling in streaming operations and added support for dynamic row reading.
- Refactored existing code to integrate with the new streaming architecture, improving performance and memory usage.
…dules

- Removed unused imports and types from `lib.rs`, including `WriteSummary` and various reader types.
- Changed visibility of several methods in `ReadOptions` and `WriteOptions` to `pub(crate)`.
- Simplified `XlsxReader` and removed the `DynamicRows` and `TypedRows` structs, consolidating functionality.
- Updated `serde_helpers` to include new serialization functions for date types.
- Modified `StreamingRows` and `StreamingTypedRows` to use updated error handling.
- Refactored `XlsxWriter` to remove `WriteSummary` and adjust method signatures for consistency.
- Updated tests to reflect changes in the API, ensuring compatibility with the new structure.
- Added `serde_json` as a dependency in `Cargo.toml`.
- Updated README files to include .NET parity contract information and testing instructions.
- Introduced a new JSON contract file for shared behavior between .NET and Rust.
- Implemented Rust tests for dynamic and typed queries to match .NET contract.
- Enhanced memory handling in the streaming path of the OOXML reader.
- Updated tests to reflect changes in expected row counts and empty row handling.
- Created new C# tests for compatibility with Rust's parity contract.
Add byte-oriented XLSX read/write support and tests, expose it through a browser WASM adapter, add the responsive web demo with Playwright coverage, and deploy the generated site through GitHub Pages Actions.
- Introduced a new GitHub Actions workflow for browser wasm builds and tests.
- Enhanced README files to document new features including inclusive A1 start/end ranges and browser WebAssembly adapter.
- Updated `ReadOptions` to support an optional end cell for dynamic queries.
- Implemented validation for cell ranges to ensure end cell does not precede start cell.
- Added new tests for inclusive cell range queries and error handling for invalid ranges.
- Updated web demo to include end cell input and corresponding functionality in the UI.
- Bumped Playwright version in web demo dependencies for improved testing capabilities.
The implementation of this property makes jagged rows being returned as they are instead of having an exception being thrown when a mismatched number of fields is found.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] - Csv read error: Column 32 not found in Row 2