Design a useful data format

[MichaelClerx]

@jonc125 Can we use a (new) combine archive variant for this?

(For example, in Myokit I use a ZIP file with a text file and a binary file, the text file describes the binary file (fields, lengths, endianness). Can even stick a reading script in it! )

[jonc125]

As noted in other issues, the COMBINE archive is really orthogonal to this question. It can contain any kind of data files.

For 'typical' data we'll need to read CSV. For multi-dimensional arrays we should probably think a bit more about what the use cases are.

Aspects to consider:

• Importing existing experimental data easily
• Presenting data easily on the web interface (plots for main protocol outputs, but potentially being able to view intermediate data in a nice way would be a useful new feature)
• Storing data compactly on disk

[MichaelClerx]

Agreed! But I think whatever we do for point 3 (compact storage), it would be a good idea to implement it within a combine format. That should be an easy way to formalise / publish what we are doing, no?

[MichaelClerx]

https://frictionlessdata.io

[MichaelClerx]

https://neurodatawithoutborders.github.io/

[MichaelClerx]

Closing for now. Should maybe have been a project_issue anyway :-)