Check of individual file sizes not accurate?

@LiNk-NY

@LiNk-NY
We are moderating a package HDO.db. In the extdata folder there is an sqllite file that is 6.1 M

shepherd@jbcj433:~/PkgReviews/HDO.db/inst/extdata(main)$ ls -lrth
total 6.1M
-rw-rw-r-- 1 shepherd shepherd 1.9K Aug 12 11:22 parse-obo.R
-rw-rw-r-- 1 shepherd shepherd 6.1M Aug 29 08:21 HDO.sqlite
-rw-rw-r-- 1 shepherd shepherd 5.1K Aug 29 08:21 get_sqlite.r

BiocCheck does not give the WARNING of files over 5M in size

─ BiocCheck results ──
0 ERRORS | 0 WARNINGS | 3 NOTES

See the HDO.db.BiocCheck folder and run
    browseVignettes(package = 'BiocCheck')

I cannot ingest the package because of git's limitation on file size which is why the individual check is so important and the ingestion script will not let me proceed

> .precheck_filesize("/home/shepherd/PkgReviews/HDO.db")
Error in .precheck_filesize("/home/shepherd/PkgReviews/HDO.db") : 
  files larger than 5Mb:
  /inst/extdata/HDO.sqlite

@vjcitn follow up to what I was saying with HDO.db -- files of larger size we normally suggest as hub packages -- granted this is an annotation package so we can make and exception and treat as a traditional annotation package but then it will not be in git.bioconductor.org and just uploaded manually as a tar.gz once it passes review

@lshep

Hi Lori, @lshep
I don't think it was written to give a warning for annotation packages. Do you want this to change?

BiocCheck/R/checks.R

Lines 232 to 249 in 9928a7c

    
           checkIndivFileSizes <- function(pkgdir) 
        
           { 
        
               pkgType <- getPkgType(pkgdir) 
        
               if (is.na(pkgType) ||  pkgType == "Software") { 
        
                   maxSize <- 5*10^6 ## 5MB 
        
                   allFiles <- list.files(pkgdir, all.files=TRUE, recursive=TRUE) 
        
                   allFilesFullName <- file.path(pkgdir, allFiles) 
        
                   sizes <- file.size(allFilesFullName) 
        
                   largeFiles <- paste(allFiles[sizes > maxSize], collapse=" ") 
        
                   if (any(sizes > maxSize)) { 
        
                       handleWarning( 
        
                           "The following files are over 5MB in size: ", 
        
                           paste0("'", largeFiles, "'", collapse = " ") 
        
                       ) 
        
                       return(TRUE) 
        
                   } 
        
               } 
        
           }

As explained in the vignette:

BiocCheck/vignettes/BiocCheck.Rmd

Lines 169 to 173 in 9928a7c

    
           * **Checking package size** 
        
             Checks that the package size meets _Bioconductor_ requirements. The current 
        
             package size limit is 5 MB for Software packages. Experiment Data and 
        
             Annotation packages are excluded from this check. This check is only run if 
        
             checking a source tarball. (ERROR)

@vjcitn

Ah that makes sense. I think it should be changed. We can't for git and we want large files to be hub hosted or server based at least that is the way we've been headed. I think the only software was legacy from when we had traditional experiment and annotation packages.
But am open to a second opinion on this @vjcitn

update docs for size limit checks for data files, closes #167 and #67

update docs for size limit checks for data files, closes Bioconductor…

Navigation Menu

Repository navigation

Check of individual file sizes not accurate? #167

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	checkIndivFileSizes <- function(pkgdir)
	{
	pkgType <- getPkgType(pkgdir)
	if (is.na(pkgType) \|\| pkgType == "Software") {
	maxSize <- 5*10^6 ## 5MB
	allFiles <- list.files(pkgdir, all.files=TRUE, recursive=TRUE)
	allFilesFullName <- file.path(pkgdir, allFiles)
	sizes <- file.size(allFilesFullName)
	largeFiles <- paste(allFiles[sizes > maxSize], collapse=" ")
	if (any(sizes > maxSize)) {
	handleWarning(
	"The following files are over 5MB in size: ",
	paste0("'", largeFiles, "'", collapse = " ")
	)
	return(TRUE)
	}
	}
	}

	* Checking package size
	Checks that the package size meets _Bioconductor_ requirements. The current
	package size limit is 5 MB for Software packages. Experiment Data and
	Annotation packages are excluded from this check. This check is only run if
	checking a source tarball. (ERROR)

Check of individual file sizes not accurate? #167

Description

Activity

LiNk-NY commented on Aug 29, 2022

lshep commented on Aug 29, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions