Renegade Public Forums
C&C: Renegade --> Dying since 2003™, resurrected in 2024!
Home » Tiberian Technologies / Blackhand Studios » Tiberian Technologies Forum » More general Database handling
More general Database handling [message #465732] Sat, 07 April 2012 22:56 Go to next message
Sonarpulse is currently offline  Sonarpulse
Messages: 18
Registered: April 2012
Karma: 0
Recruit
Currently, Westwood3D can handle two "layers" of object databases: temps.ddb for map-specific changes, and objects.ddb for game-wide presets. This aproach I fine rather limiting. For example, if someone is working on two seperate mods that share some assets, or if one is making a map for a game like APB, whose presets are constantly changing, they must manually clone presets from one database.

Ideally, I think it would greatly enhance w3d if an arbitrary number databases could work as patches a là temps.ddb. So for example, somebody could make a mod pacalge that would inherit changes to the underlying game, and contain both mod-package-wide and map specific "temps.ddb". This would allow for presets to be managed in many more ways, giving much-needed flexibility to developers.

Naturally this same functionality could be also applied to string databases, allowing among other things translators to update their translations with much greater ease. (And partial/old translations to still work.)


http://i285.photobucket.com/albums/ll51/Sonarpulse/Forum/CnP.png
Re: More general Database handling [message #465826 is a reply to message #465732] Mon, 09 April 2012 15:53 Go to previous messageGo to next message
StealthEye is currently offline  StealthEye
Messages: 2518
Registered: May 2006
Location: The Netherlands
Karma: 0
General (2 Stars)

This would be nice in principle, but not a priority for TT to do imo.

BlackIntel admin/founder/coder
Please visit http://www.blackintel.org/
Re: More general Database handling [message #465846 is a reply to message #465826] Mon, 09 April 2012 22:00 Go to previous messageGo to next message
Jerad2142 is currently offline  Jerad2142
Messages: 3805
Registered: July 2006
Location: USA
Karma: 6
General (3 Stars)
Maybe you guys should just use temps to ship updates back and forth, turn the temp files into objects presets and then delete the temp file, allowing you to create another new temp file for further updates.

Re: More general Database handling [message #465847 is a reply to message #465732] Mon, 09 April 2012 22:14 Go to previous messageGo to next message
Sonarpulse is currently offline  Sonarpulse
Messages: 18
Registered: April 2012
Karma: 0
Recruit
What happens if you drop a temps.ddb in /data? I have never tried it, but I assume it wouldn't work.

http://i285.photobucket.com/albums/ll51/Sonarpulse/Forum/CnP.png
Re: More general Database handling [message #465849 is a reply to message #465847] Mon, 09 April 2012 22:25 Go to previous messageGo to next message
Jerad2142 is currently offline  Jerad2142
Messages: 3805
Registered: July 2006
Location: USA
Karma: 6
General (3 Stars)
Not sure if that works, but you could drop the presets in data after you update them; thus allowing you to bypass re-exporting always.dat and all that fun stuff.

Re: More general Database handling [message #465859 is a reply to message #465732] Tue, 10 April 2012 02:07 Go to previous messageGo to next message
danpaul88 is currently offline  danpaul88
Messages: 5795
Registered: June 2004
Location: England
Karma: 0
General (5 Stars)
Re-exporting always.dat isn't that bad really, only takes a minute or two and using the MIX patcher I developed for BHP (and any other mod team that wants it) patches to always.dat (and other .dat / .mix files) can be shrunk down to the size of the modified / new files within the MIX archive only.

AR have been doing this for our patches for years, typically our minor patches come out at about 30mb, even though they update the always.dat file (currently 400mb+).


http://steamsignature.com/card/1/76561197975867233.png

[Updated on: Tue, 10 April 2012 02:08]

Report message to a moderator

Re: More general Database handling [message #465873 is a reply to message #465732] Tue, 10 April 2012 07:23 Go to previous messageGo to next message
Sonarpulse is currently offline  Sonarpulse
Messages: 18
Registered: April 2012
Karma: 0
Recruit
Well you can already use a always2.dat to make a dynamic patch (or your superior method to make a static patch). But I am just trying to patch objects.ddb. The basic reason I see patches as superior to overwrites is the "upstream" changes can be inherited with a patch.

In short, what your mix patcher and always2.ddb achieve on the archive level, I want to acceive on the (preset) file level.

Edit: But perhaps making a tool to combine to preset databases, as opposed to an engine modification to do that on he fly would be easier and safer. Is all the code for the TT preset and string database editors public? It would be an interesting feature to try to implement myself.


http://i285.photobucket.com/albums/ll51/Sonarpulse/Forum/CnP.png

[Updated on: Tue, 10 April 2012 07:46]

Report message to a moderator

Re: More general Database handling [message #465880 is a reply to message #465732] Tue, 10 April 2012 09:22 Go to previous messageGo to next message
jonwil is currently offline  jonwil
Messages: 3555
Registered: February 2003
Karma: 0
General (3 Stars)

strings.tdb stuff is public.
objects.ddb code is not public (for various reasons that I cant go into) and isn't usable enough yet to be able to combine multiple preset files into a single one although its on the wishlist.


Jonathan Wilson aka Jonwil
Creator and Lead Coder of the Custom scripts.dll
Renegade Engine Guru
Creator and Lead Coder of TT.DLL
Official member of Tiberian Technologies
Re: More general Database handling [message #465882 is a reply to message #465732] Tue, 10 April 2012 09:50 Go to previous messageGo to next message
Sonarpulse is currently offline  Sonarpulse
Messages: 18
Registered: April 2012
Karma: 0
Recruit
Well maybe I wil just take a look at the string editor then. It won't help my project, but will help translators. And if there any similarities in the way strings and presets are handled, maybe it will help you guys.

Glad to hear it's on the wishlist.


http://i285.photobucket.com/albums/ll51/Sonarpulse/Forum/CnP.png
Re: More general Database handling [message #467396 is a reply to message #465732] Wed, 16 May 2012 01:15 Go to previous messageGo to next message
Sonarpulse is currently offline  Sonarpulse
Messages: 18
Registered: April 2012
Karma: 0
Recruit
Not to bump my own topic, but I was just thinking, why don't you guys scrap the fancy binary object and string database formats, and just make the game read text files? hell the logic is already half there with the LE and tdbedit import and export functions.

Us modders would have the a benefit of a much easier-to-use format (especially because importing the text exports isn't always safe), and you guys would be spared from having to develop extra database editors.


http://i285.photobucket.com/albums/ll51/Sonarpulse/Forum/CnP.png
Re: More general Database handling [message #467493 is a reply to message #467396] Fri, 18 May 2012 15:08 Go to previous messageGo to next message
saberhawk
Messages: 1068
Registered: January 2006
Location: ::1
Karma: 0
General (1 Star)
Sonarpulse wrote on Wed, 16 May 2012 01:15

Not to bump my own topic, but I was just thinking, why don't you guys scrap the fancy binary object and string database formats, and just make the game read text files? hell the logic is already half there with the LE and tdbedit import and export functions.

Us modders would have the a benefit of a much easier-to-use format (especially because importing the text exports isn't always safe), and you guys would be spared from having to develop extra database editors.


That is never going to happen for many reasons. One of the big ones is the huge number of bugs that will be caused by replacing such a vital system, another is that *everything* would immediately be incompatible with 4.0.
Re: More general Database handling [message #467509 is a reply to message #467493] Sat, 19 May 2012 00:01 Go to previous messageGo to next message
Sonarpulse is currently offline  Sonarpulse
Messages: 18
Registered: April 2012
Karma: 0
Recruit
Well, as to incompatability, you could always keep the ability to read the binary formats and export their text counterparts. But, I can't really say anything to the issue of bugs. Thanks for letting me know.

Edit: While I've bumped this thread I guess I'll fire away with another question: Do any presets still need GDI or Nod in their name for hardcoded reasons, and if so can that easily be changed?


http://i285.photobucket.com/albums/ll51/Sonarpulse/Forum/CnP.png

[Updated on: Sun, 20 May 2012 01:00]

Report message to a moderator

Re: More general Database handling [message #469450 is a reply to message #465732] Mon, 18 June 2012 11:53 Go to previous messageGo to next message
Sonarpulse is currently offline  Sonarpulse
Messages: 18
Registered: April 2012
Karma: 0
Recruit
Aha! the new ddbedit can now dump to a text file! Will RC 1 (or beta 6) bring the ability to re-import those dumps? Thanks so much, this new text file format is far superior than LE's tab delimited export (and LE import never seems to work). Having ddbedit import will effectively mean my dreams for workable text-based object handling are realized, as it doesn't matter what w3d actually uses if you can always mod presets as text.

On another note, I noticed tdbedit now has a menu item to import a preset database. Does that mean the tools are going to be combined?

EDIT:
  1. Comments: To make this a truly robust format it would be nice of have comments. It's basically an INI but with XML style end tags, so I think semi-colon comments would be sufficient.
  2. ID references: It would be super convenient if preset numerical IDs were also in the ddb dump, that way presets that point to other presets can be easily determined. If it's too much of a pain to make them editable, just make them a comment at the beginning of a preset.
  3. ID dereferences: This would be the icing on the cake: every time a preset points to another preset, put the name of the referenced preset as a comment next to the referring field (and ID).


http://i285.photobucket.com/albums/ll51/Sonarpulse/Forum/CnP.png

[Updated on: Mon, 18 June 2012 12:06]

Report message to a moderator

Re: More general Database handling [message #470103 is a reply to message #465732] Wed, 27 June 2012 20:22 Go to previous messageGo to next message
Sonarpulse is currently offline  Sonarpulse
Messages: 18
Registered: April 2012
Karma: 0
Recruit
Ok, not to be a whiny, begging-for-features, post-bumping ass. But is there any chances that there will be any sort of working text import for presets? Even without all those fancy features I hoped for?

I am working on a mod: https://www.bluehellproductions.com/forum/index.php?showtopic=25411 that is mainly rigging, so the availability of text import greatly affects my workflow.


http://i285.photobucket.com/albums/ll51/Sonarpulse/Forum/CnP.png

[Updated on: Wed, 27 June 2012 20:25]

Report message to a moderator

Re: More general Database handling [message #472427 is a reply to message #465732] Tue, 31 July 2012 00:42 Go to previous message
jonwil is currently offline  jonwil
Messages: 3555
Registered: February 2003
Karma: 0
General (3 Stars)

Adding numerical preset IDs to the ddb dump, that I will do for the next version. Resolving references is tricky due to how the dump code works.

Making a re-importer, that wont be happening anytime soon due to how complex such a thing would be.

And the reason tdbedit.exe reads objects.ddb is so that it can use the sound definitions in it to provide better editing for sounds attached to strings.

EDIT: Turns out our code already resolves numeric IDs to preset names e.g. KilledExplosion=Regular_Explosion or Engine Start Sound=Supply_Truck_Start


Jonathan Wilson aka Jonwil
Creator and Lead Coder of the Custom scripts.dll
Renegade Engine Guru
Creator and Lead Coder of TT.DLL
Official member of Tiberian Technologies

[Updated on: Tue, 31 July 2012 01:17]

Report message to a moderator

Previous Topic: Server Crashdump
Next Topic: delete plz
Goto Forum:
  


Current Time: Thu Apr 18 01:02:23 MST 2024

Total time taken to generate the page: 0.00970 seconds