Testers wanted: STBLizer (updated 26/9/09, new templates)
 

This is a simple tool that allows you to create STBLs by batch-importing one or more CSV files. It's really fast - I've tested this tool on a EEE PC 701 running Windows Server 2003, and it imported 9,746 strings from 42 different files in under 10 seconds! Take that, underpowered netbook haters.

The latest version lets you use keys in string form as well as hash form. If you understand how localization works you can use this add new strings to the existing systems, or you can use STBLizer as a basic STBL compiler for script mods and such.

Requirements
Microsoft .NET Framework 3.5
If you want to replace the default game strings, you'll also need will an unpacked language STBL file, extracted with a package editor like s3pe or Postal. After the 1.3.24 patch, they are now located in GameData\Shared\Packages\DeltaBuild0.package. This is an optional step: if you don't want to edit the original game strings, you don't need to extract one.

Installation
Unzip it wherever you like.

1.3 changes

• Updated s3pi library to latest version (to fix duplicate string bug)

1.2 changes

• Now accepts keys in string form (e.g. "Gameplay/Excel/Careers/CareerList:Science").
• Updated s3pi library to latest version

1.1 changes

• Removed the filename boxes (since loading a STBL is no longer compulsory)
• Now compiles new STBLs from scratch, instead of updating the 4MB main one (thanks to Tiger's learnings). Export functionality is still retained, so you can dump the game STBL to a file for editing in the first place.
• Optimised the key parsing code
• Fixed bug with Dictionary.Add
• Changed name from TS3 String Importer to STBLizer

Usage
First of all, select your STBL and click "Export STBL to CSV" to save its contents to a plaintext CSV file.

Copy and paste the entries you want to edit into a separate CSV and change the strings there (don't edit the hash keys). Then go back to STBLizer and click "Compile new STBL". Select your CSVs (you can select multiple files at a time), and the tool will build the STBL. Save the new STBL somewhere, import it into a .package with the correct type (0x220557DA) and instance (see below), put it in your Mods\Packages\ folder, and you're good to go.

How do I edit CSV files?
If you feel intrepid, use Notepad. If you want the familiarity of a spreadsheet program, Openoffice.org Calc is terrific.

But, whatever you do, DON'T USE MICROSOFT EXCEL TO EDIT CSV FILES! Excel strips out ALL the double quotes around text, which will break any multi-line strings and confuse the String Importer! That's why CSVs exported with this tool have quotes around every field by default.

Example snippet from a valid CSV

(*will not actually work without an XML hack)

Formatting notes

• The order of the strings does not matter.
• The tool will ignore any fields past the first two, but the first must contain a valid key and the second a valid string. Strangeness may result if the second column is empty (you might get the **** MissingText **** error).
• This tool does not check for edits to the same string in different files. There Can Only Be One, so the file that loads last will overwrite any entries changed by previous files.
• The most common error you will get is if the CSV is badly formatted (missing quotes or commas, or a blank line where there shouldn't be one). The log box will show the progress of the CSV reader around the point the file "breaks", so find that point and fix the file then try importing it again.
• The csv files should be encoded as UTF-8, otherwise curly quotes and special characters will show up as garbage. If you decide to use Calc, make sure you select "Unicode (UTF-8)" in the dropdown 'Character set' box when you open up the CSV. If using Notepad, you can choose encoding when you save.

Instances
You must choose a unique instance for your STBL when you package it, following these guidelines:

The first byte must be the appropriate localisation code for your game. There is a list here. 00 is English.
The next 3 bytes are random.
Finally the last 4 bytes must be less than 0BA8342F, the code for the default STBL, so that the game will prioritise them. (I hope you can read hex!) That means anything from 00000000 to 0BA8324E will be accepted by the game engine. Everything above will be rejected.

In summary:



Default text replacements
I've attached to this post two zips containing template .csv files. The "templates" are really just the files containing all the relevant strings for a particular theme: one zip contains the last/male/female names (used to make this mod), while the other contains the names and job descriptions for all default careers. They should work with non-English versions, provided you choose the appropriate language code when you package the .stbl.

For names, you will need 1010 female names, 1037 male names, and 3127 last names to *completely* overwrite the EA names. After you have the names, make a fresh copy of the .csv template you want to edit, open it up for editing, and replace crappy EA names with your cool ones. The reason there are 6254 last names because they are doubled as male/female pairs. It will be necessary to duplicate the list and sort the names alphabetically (at least this is what I did). Now Excel *may* work as an editor for this job - only because you're working with names, not strings that could contain illegal characters (, or ") or strings with line breaks in them. However, you'll have to work out encoding problems on your own.

The same goes for the career templates. Just remember if you want more than a superficial text change, you'll have to make an XML mod.

What happens if the string also contains quotes?

Straight double quotes are represented by two consecutive quotation marks:



However most (but not all) of the EA text uses curly double quotes, counting as separate characters. Single quotes are safe.

2 questions:

1) does the stbl have to get put back in a package for the game to care? (haven't played with STBL changes yet)

2) if you don't mind sharing your source code (or providing a .dll) I could put a snazzier interface on that so users could edit the CSV file right in the program (with handy features like built in search).

I'm still a bit of a C# novice but UI's are my specialty

1) Yes you have to package it with the right type/instance (0x220557DA/0x000F16B00BA8342F) for the changes to show up in-game.

2) If you like. I haven't considered doing an editing interface at this point; I just wanted to get the tool up and running first. Plus it's really designed so you can maintain multiple CSV files (for modularity) rather than edit directly The One STBL. Still you're welcome to have a look at the source.

Well that's what I meant, just editing the CSV files inside the program thereby making it impossible for people to screw them up. I'll take a look, burnt myself out today messing with LINQ to XML on my own project so I'll peek at your code tomorrow or something

Actually that sounds like a good idea. Go ahead, exploit the code as you wish

Took a look at the code, shouldn't be too hard to splice it into a new app. I've made a mockup for the UI:



Doesn't do anything yet, but basically the .csv files will go on the left, and you can then select one to edit it in the right side. The big box at the bottom is for editing long strings that don't fit nice in the right side editor. It doesn't start out that big but I took the screen after playing with the resizing bars (white) and didn't feel like taking another one :P.

What's left is splicing in your code so that buttons do things .

Update:

I've got it now so that it finds all the .csv files in your packages directory and lists them, and it also loads them into the editor. The editor doesn't actually edit yet since there are some kinks in the loading to work out. Once that's worked out the next step will be saving back to CSV, and then merging the CSVs back into the STBL (not too hard since that code is mostly written by channTL), and hopefully from there automatically packing that back into a .package.

That way people can just put a strings.csv file in with any mod they distribute, and the user only had to run the StringEditor and it can do one click magic to make a merged STBL of all the .csv's.

Why can't the mod creators just put a package in there with the string tables already loaded, then people using the mod just have to put the package in their mods directory. As long as each package has a unique instance ID for their string tables, any number of packages can define new or replacement strings.

I thought that was impossible!

I experimented with it a while ago with a 2-string STBL written in hex. If the new STBL has the same instance as the original (000F16B00BA8342F), the rest of the game text goes missing with asterisks. If it has a unique instance, the changes don't appear at all.

When you tested, did you make sure it still had the right language marker? (IE the high byte of the instance was 00 for english)

Actually I just dug out my test STBL and tried it again with a random instance, 0x0028C7266C27A5A8. No changes appeared. I even tried it as an override mod, no effect there either.

Hmm, very odd, when I just tried it, it worked perfectly fine...

Ok I just tried again with a newly made package, and this time it didn't work...

Are you adding new strings or replacing them? Was there something you did different? There must be some secret to all of this!

Purely replacing. There are a few difference and I'm trying to isolate them now... But so far nothing. I'm wishing I hadn't deleted that package now.

Right then, I have an answer.

It seems like it picks them in order of the lower 32 bits, going from lowest to highest. So to override just a part of the standard string table, you have to have it LOWER than the standard has.

That means it has to be below BA8342F to work. BA8342E will override strings, BA83430 will not. Bits 56..32 do not matter and can be any random value. Of course bits 64..56 must be the language code.

For once I'm glad I had a bug. The reason my first test worked is because I was trying to mask off the language code, but I accidentally masked it off using a 32 bit value instead of a 64 bit value, so bits 56..24 were all 0, meaning it fell below the value from the standard strings table. If I hadn't had that bug, I probably would have never figured out that it WAS possible to replace strings.

At this point the core editing functionality is complete -- all that's left is geting STBL's out of and back into .package files. I may or may not try to tackle that tomorrow.

OK, I gave this another try and changed the instance on my test file to match. No luck AGAIN. Am I interpreting the above correctly? Based on the default STBL instance:

00 = language code
0F16B0 = random bytes
0BA8342F = must be lower than this

That's how I'm doing it. My latest test was using 0x00FFFFFF0AFFFFFF.

Example: here

ETA: And looking at the instance IDs from a number of EA string tables, it appears this is by design, and not just a quirk in how it works. All the store stuff the low 32 bits are 8352xxxx

Your package works fine for me, which is even more puzzling in a way. There must be some weird problem on my end.

Well which strings were you replacing in your test?

I'm using the name list from my mod mainly, also I've tried replacing some of the UI strings as you did to avoid needing to load up a save. I've been using Pescado's stblc to compile the .stbl files.

EDIT: OK, I finally got a unique instance STBL to load, albeit a copy of the full one. Tinkering continues.

EDIT 2: A breakthrough! I finally got my name mod to work with a new instance and have uploaded a new version of it. Only took about a week to work out what was wrong

Anyway, as a side effect of this work I've upgraded the TS3 String Importer to compile new STBLs from CSVs instead of just importing them. Now the tool is even more user-unfriendly I will upload this newer version soon after I streamline it a little more.

Non-English letters
 

Thanks a heap for a great mod and a good explanation how to do this. After coming up with Swedish names, it was easy as pie to get the package together, and all new Sims get names from my lists now.

There is one problem however. The Swedish vowels 'å', 'ä' and 'ö' doesn't seem to be added correctly into the package. In the csv files everything looks okay, but in the game the letters are substituted with other characters. The 'ä' is, for example, replaced with an underscore, making a name like 'Hägerstad' come out as 'H_gerstad' in the game. When I look at the package values in s3pe the same name looks like this: 'H�gerstad'.

Do you have any suggestions as to how I can get around this? I am, sadly enough, utterly clueless as to what to do. I used this type of mod in Sims2 as well, and it worked fine, so I've never had to muck about with this issue before...