|
Socoder -> Off Topic -> SQLite Database |
|
| Fri, 29 Mar 2013, 16:27 | |
Cower
|
I think I’ve decided to go with using SQLite databases as the main format for most of the stuff my engine loads. Easier that way, especially since I can merge them all together into one giant file later.
|
| Fri, 29 Mar 2013, 16:27 | |
|
Cower
|
Yep, going with SQLite. Wrote a nice interface to work with it in C++11 and it's quite a bit easier to load data in thanks to it.
|
| Fri, 29 Mar 2013, 16:41 | |
Jayenkai
|
What!? You're not using PNGs to store data!!? What kind of craziness is this!?!!
-=-=- ''Load, Next List!''
|
| Sat, 30 Mar 2013, 02:42 | |
|
Cower
|
I'm using PNGs/other images and my own model format for data like that, but for more flexible data that would be a pain to read in/out in a binary format, DBs are admittedly a lot better. All I really have to do is open a DB, pass it to something to read in data, and then go on using the DB for whatever I want. No need to worry about file position or alignment or endianness (of non-blobs, anyway), since SQLite handles it all. Basically, it's a matter of avoiding what would be a huge pain. I had four options, more or less, for a lot of data: JSON, Sparse (my own format), some proprietary binary format I haven't created yet, or databases. Turns out databases are smaller than JSON, easier to parse than both JSON and Sparse (though Sparse is very fast to parse), and the binary format doesn't really win anything because I can't just read it in due to alignment issues, endianness, etc., again. Also, for an example, this is a screenshot of a font loaded in from an sqlite database. Linked because it's enormous. The loading code itself is only about 80 lines, not counting some variable declarations and checking to make sure the database is open.
|
| Sat, 30 Mar 2013, 06:26 | |
Afr0
|
You're going with SQLite as the... main format for most of the stuff your engine loads? Care to explain? o.O Are you going to store all your data uncompressed and then use *.sqls to map them? Or are you going to load stuff into a SQLite DB in memory? That makes sense, but how do you store stuff on disk? ... thanks for creating this topic. It's the most interesting topic we've had for ages, as Socoder seems to have debased into some kind of Learning With Monkey/NSFW/Technological News hybrid. Hardly any coding topics anymore, certainly not any good/challenging ones... -=-=- Afr0 Games Project Dollhouse on Github - Please fork!
|
| Sat, 30 Mar 2013, 12:19 | |
|
Cower
|
The databases are kept as a series of database files. It's not stored as an SQL file or kept in memory, though I can create temporary in-memory DBs. The in-memory DBs aren't really useful except for probably some odd cases, like cvars (since SQLite columns are dynamically typed, it makes it easy to just keep a single cvar table of string keys and arbitrary values). The advantage is that database files can be arbitrarily merged, so I can reasonably sort things as needed. Since all resources are accessed over read-only DB "connections" (term loosely used since a connection is just disk access to the DB file), it's feasible to create DB connections as needed and pass those off as needed for reading data. For an example of a DB file, this is a font database I've used in Snow for testing: dl.dropbox.com/u/31892/snow/fonts.db You can open it using sqlite3, etc. The comparable JSON files are 1.1MB when minified, the database is only 569kb. A single ASCII JSON font file will be about 6kb less than the database. A single JSON file of a variety of Unicode characters is much larger, mainly due to the kernings map. The same fact about kernings also applies to the database, though it obviously scales better. In all practicality, I don't intend to support more than a few ASCII fonts (zero support for non-English character sets, really, since I don't need them - if I did, I'd load non-English glyphs on demand), so this ends up working out fairly well for the few fonts I do need. For an idea of how loading something from one of the DBs works, this is the font loading code:
|
| Sat, 30 Mar 2013, 18:16 | |
|
Afr0
|
 The databases are kept as a series of database files. It's not stored as an SQL file What, what? You designed your own format for use with SQLite? -=-=- Afr0 Games Project Dollhouse on Github - Please fork!
|
| Sat, 30 Mar 2013, 18:21 | |
Cower
|
SQLite's format is databases, not SQL files.
|
| Sun, 31 Mar 2013, 05:59 | |
|
Afr0 |
SQL files are databases...
|
| Sun, 31 Mar 2013, 10:52 | |
|
Cower
|
No, they aren't. An SQL file is a series of SQL statements in text.
|
| Mon, 01 Apr 2013, 02:03 | |
|
Afr0
|
I only ever worked with SQLite in memory, that's probably why.
-=-=- Afr0 Games Project Dollhouse on Github - Please fork!
|
|
|
| |||
|
| ||||