Skip to content

add occupancy bookkeeping class and HIPO banks - #1378

Draft
baltzell wants to merge 52 commits into
developmentfrom
occupancy
Draft

add occupancy bookkeeping class and HIPO banks#1378
baltzell wants to merge 52 commits into
developmentfrom
occupancy

Conversation

@baltzell

@baltzell baltzell commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

To reduce the calibration data volume from detector hit banks, this adds new occupancy bank schemas, thread-safe OccupanceTable book-keeping class with I/O helpers for standard indexed HIPO banks, and OccupanceEngine ready for all standard clas12 detectors.

The new occupancy banks currently look like this:

        "name": "OCC::FTOF::adc",
        "group": 500,
        "item": 14,
        "info": "Occupancy Bank",
        "entries": [
            { "name":"sector"    , "type":"B", "info":"sector"},
            { "name":"layer"     , "type":"B", "info":"layer"},
            { "name":"component" , "type":"S", "info":"component"},
            { "name":"hits"      , "type":"F", "info":"number of hits"}
  • Finalize occupancy bank naming
  • Bank group number
  • Use FilteredBank for order-hijacking by denoising
  • Check index count for all detectors
  • Check for missing detectors
  • Ignore bank rows with negative indices
  • YAML variable, occupancyPrescale

The main consumer of these new occupancy banks is clas12-timeline.

Depends on #1380, #1381, #1382, #1383.

@baltzell
baltzell force-pushed the occupancy branch 4 times, most recently from 7df1707 to bf9ad64 Compare August 12, 2026 21:00
@baltzell baltzell changed the title add occupancy class add occupancy bookkeeping class and HIPO banks Aug 13, 2026
@baltzell
baltzell marked this pull request as ready for review August 13, 2026 22:21
@baltzell
baltzell marked this pull request as draft August 14, 2026 21:28
@baltzell
baltzell marked this pull request as ready for review August 18, 2026 18:00
Comment thread etc/bankdefs/hipo4/occupancy.json
@baltzell baltzell added the calib label Aug 25, 2026
@baltzell

baltzell commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator Author

Negative indices definitely doesn't work:

java.lang.IllegalArgumentException: Index value out of range (0–65535) : -119
	at org.jlab.utils.groups.IndexedList.validateIndex(IndexedList.java:86)
	at org.jlab.utils.groups.IndexedList.add(IndexedList.java:99)
	at org.jlab.utils.groups.IndexedTable.addEntry(IndexedTable.java:95)
	at org.jlab.detector.calib.utils.OccupanceTable.fill(OccupanceTable.java:82)

Probably this means the addEntry isn't being used for translation tables in the decoder (there's a string-based alternative) ....

@baltzell

baltzell commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator Author

Nope, IndexedTables in decoding always have positive indices (they're defined only by CCDB tables), and addEntryFromString has the same issue with negatives. The negative is created on bank writing.

Note, the swaps service will throw an exception for RICH's layer>128 hits.

@c-dilks

c-dilks commented Aug 25, 2026

Copy link
Copy Markdown
Member

Some info about group and item here on pp. 11 ff. of Mathieu's HIPO spec writeup:

https://code.jlab.org/ouillon/hipo4j/-/raw/691447f5a212a23a9561055cb9985e104828794f/specification/hipo-file-format.pdf?inline=false

(CC @raffaelladevita)

table.addEntry(index);
table.setDoubleValueByHash(0.0d, 0, hash);
}
table.setDoubleValueByHash(table.getDoubleValueByHash(0, hash) + weight, 0, hash);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be *weight?

"entries": [
{ "name":"sector" , "type":"B", "info":"sector"},
{ "name":"layer" , "type":"B", "info":"layer"},
{ "name":"component" , "type":"S", "info":"component"},

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to keep order for FTOF, CTOF, CND, ...?

{ "name":"hits" , "type":"F", "info":"number of hits"}
]
}
]

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CND is missing

@Override
public boolean processDataEventUser(DataEvent event) {
for (OccupanceTable t : tables) {
RawDataBank b = new RawDataBank(t.getHitBank(), 1000, OrderGroups.NOMINAL);

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we may want NODENOISE instead of NOMINAL

b.read(event);
t.fill(b, false);
}
if (++events % prescale == 0) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this thread safe?

* @param event
* @return
*/
public DataBank create(long events, DataEvent event) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in multithreading, is it possible the table gets modified by thread X while thread Y is creating the bank?

@baltzell
baltzell marked this pull request as draft August 27, 2026 23:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants