Skip to main content
Information on cookies
On the basis of your freely given consent, which can be revoked at any time, your visit to our website is recorded using Matomo, an open source web analytics software program. The information produced will be used solely for statistical purposes and to improve the website and server. No personal data will be stored or shared with third parties. For more information, please refer to the Privacy Policy on our website. By clicking “Accept”, you consent to our use of cookies for analytical purposes. If you do not agree to this, please click “Reject”. In principle, you can visit our website without cookies being enabled. This does not apply in the case of essential cookies.

A Long and Lasting Love File Format

ERROR: Content Element with uid "689" and type "dce_text_with_icon" has no rendering definition!

ERROR: Content Element with uid "670" and type "dce_lightbox" has no rendering definition!

ERROR: Content Element with uid "671" and type "dce_readonmydear" has no rendering definition!

ERROR: Content Element with uid "675" and type "dce_lightbox" has no rendering definition!

ERROR: Content Element with uid "677" and type "dce_readonmydear" has no rendering definition!

ERROR: Content Element with uid "685" and type "dce_readonmydear" has no rendering definition!

ERROR: Content Element with uid "686" and type "dce_lightbox" has no rendering definition!

ERROR: Content Element with uid "687" and type "dce_readonmydear" has no rendering definition!

Problems with data infrastructure and file formats

With the rise of computers, we have seen a large increase in the size and the complexity of the data mathematicians use. In the past, we only had books like the atlas of finite groups in which some data was written down, but nowadays a database like the LMFDB, for example, contains multiple Terabytes of data.

And more data gets produced every day! Papers with new results get published at a very high speed and we currently simply do not have the infrastructure available for supporting the huge influx of data that comes with it. There are several aspects to consider here:

  • If we make a database of objects of a certain type available, other researchers will use this data to produce new results. But we want to make sure that this classification is complete, i.e. that we have all of the objects in our database (like we hopefully convinced you there are only 5 platonic solids) and that the data is correct. But how can we ensure this? Sometimes computations can take months or years to complete. And it's very easy to make a small mistake in the code written to perform the computations. Especially if this code is very complicated.

  • And what if the data was produced using closed-source software? If I can look at the code I can convince myself that the way the data was produced makes sense mathematically. But if I can't... anything could have happened behind the scenes. The authors could've written the code in such a way that it outputs exactly the results they want to see as output in the cases they describe in their paper and simply spit out nonsense any other time.

  • We want other researchers to use (or reuse) the data that has been computed. But will they actually be able to find it? With the huge influx of papers, it is sometimes difficult to get an overview of everything that is out there. Additionally, authors of papers are not always aware of best practices concerning the publication of data. A common scenario is the following: An author puts the data on their own website. They move to a different university after a few years. The website gets deleted. The data is gone forever and other people will need to redo the computations.

  • Who is going to pay for storing computed data? If the data is just a couple of MB it's probably not too hard to find a place to put it, but what if it is several Gigabytes?

  • In other areas of sciences, most data will consist of lists of numbers. Mathematical objects however can get very complicated very quickly and one therefore has to think carefully about how to store it. Especially if you want other people to be able to load the objects again in their favorite software package (which might not be the one you used for your computations). Currently, there is no universally agreed-upon standard for this. Think about polynomials, for example. A polynomial in one variable x with rational coefficients is simple enough. But what if the polynomial was defined over a field extension L of QQ? Then we would also need to store the field L. But there is no canonical way to define the field L. We need to choose a specific polynomial defining the field extension L over QQ and store this with the data. But maybe L was defined as a field extension over a field that was not QQ. And then you need to deal with it differently again. And how do you store a polynomial with multiple variables? Do you consider x^2 +x*y to be a polynomial in y over the ring k[x]? Or is it an element of k[x,y]? There are many more questions like this that could be asked and it is not easy to find a file format that is universally accepted by everyone.

Within MaRDI we work on possible solutions for these problems with a special focus on mathematical research data. Please subscribe to our newsletter to keep up to date on our findings!

 

ERROR: Content Element with uid "691" and type "dce_featured" has no rendering definition!