Calculate XPT row_count using metadata and final chunk #315

Activity

1. DmitryMK commented on Mar 10, 2025

   

   DmitryMK

   on Mar 10, 2025

   More actions

   Hi Gerry. Not proposing a solution, but I faced the same problem when implementing XPT v5 parser in JS. I calculated it from metadata using very similar approach to what you described above (here is my solution for reference
   https://github.com/defineEditor/xport-js/blob/c8da602428b25c94187befb20928e35a48d518ae/src/classes/member.ts#L162)
   Maybe the only difference that in my case I had to calculate records_start_offset manually and instead of subtracting a padding, I just used floor function.

   Mainly I just want to confirm that this approach works and gives a correct row_count at least for XPTv5, so I hope it will be implemented in ReadStat at some point.

   👍React with 👍1Reacted by gerrycampion
2. evanmiller commented on May 23, 2025

   

   evanmiller

   on May 23, 2025

   Contributor

   More actions

   For XPT v8 we should be able to just read in the observation count described here: #322
3. evanmiller

   added

   SAS

   enhancement

   on May 23, 2025
4. sasutils commented on Mar 23, 2026

   

   sasutils

   on Mar 23, 2026

   More actions

   Note that the the last record of the xport file for a dataset is padded with blanks to fill the 80 blocks used by xport files. When the data record length is less than the 80 bytes then it is possible for there to be room for extra records in that padding. In those cases SAS will ignore data records that are all blanks at the end of the last block of the file when reading the file.

   One way to detect these is to decrement your calculated number of observations for every all blank data record detected at the end of the last block.

   You can see that in action in this improved SAS macro for reading xport files.
   (https://github.com/sasutils/macros/blob/master/xport2sas.sas)

        if obslen<80 and nobs>0 then do;
         j=mod(nobs*obslen,80);
         if j=0 then j=80;
         do j=j+1-obslen to 1 by -obslen;
           if substr(lastrec,j,obslen) ne ' ' then leave;
           else nobs=nobs-1;
         end;
       end;