This is great!!! Thank you!

Just wanted to say thanks to @plasticbox for posting this link. Appreciate it!

(I understand @scumbumbo 's reasons for going with s4pe, but, I use s4s. Updated s4pe link = WIN.)

Is there a way to add the option to save the scanned file results? I would love to be able to send mine to plasticbox as most of the conflicts/duplicates and conflicting resources are the ones from plastic box like counters and overrides....

Don't worry, I know what is in my packages. If you have questions about an upload please ask in the upload comments. Please read the upload descriptions before you do so.

This goes for content by other people as well, of course.

Here is a more in-depth explanation on the two pbox packages that @scumbumbo is citing as examples in the upload description:


Files with Conflicts ‒ 1:



These are two packages from this upload (BlandCo Overhaul). They are available in two versions; quoting from the upload description:



The conflicting resources are the COBJ resources that are contained in both versions; one has the original catalogue swatch colours and one has colours that match my override. People who didn't read the upload description and installed both will have a conflict there, rather expectedly.



Files with Conflicts ‒ 2:



Not much to add to that ‒ as one might guess from the package name, the 1.4.83.1010 one is made for a specific game version (and later). This is from this upload (Flowerbox Fixes); the upload description also says that this has been updated a few times and the old versions are obsolete (unless you have a really old game, in which case you don't need the newer versions).

Hi again! I'm the original poster that it wasn't working on my Windows 10. I'm still not sure what the problem was, but your workaround worked perfectly, scumbumbo, so thanks so very much!



Just to clarify, I shut down my PC nightly and restarted post-install so that wasn't the issue. It's highly possible it could have been some other weird issue though. Any way, thanks again!

Just to clarify - the regular folder select dialog is now working for you and you don't have to check the "fallback" option? If that's working for folks I can remove that option, as I can't imagine anyone who would prefer that folder selector if they had the choice!

I should probably clarify that in the mod description - wasn't intending to pick on your mods, pbox but I just happen to have an awful lot of them and as you pointed out some of them shouldn't even be there anymore!

Don't worry, you picked good examples =) I just wanted to explain it some more so that people don't post about it umpty-five times. I tagged you because I thought you might want to link to that post from the upload description, for those who want to know more. Finding duplicates/conflicts is one thing (and your tool does a great job at it), interpreting what they mean is a whole other story ..

(A lot of my other mods have duplicates too, that's on purpose .. I have no problem explaining stuff in detail, but for anything that you don't explicitely cite, people would need to ask in the actual upload thread and not here. I have feeling you're also not keen on opening a help forum here =P )


Another thing I wanted to ask, or two rather:

* would it be possible to also make this check for "conflicts" between mods and the game files? There seems to be some CC in existence that "stealth overrides" some stuff, like textures, but since it's overriding with identical copies of the originals that's hard to find (since one does not see any effect in game, except that it breaks other overrides). I don't know what it is, I only know about it because I've had some reports that texture overrides I had made were not working unless people removed their other CC, which seems to mean they have another override that they don't know about.

* is it possible to list resources where the content is identical but not the ID? As in, identical (but renumbered) copies of the same thing (image resources in particular)? That accounts for 90% of all file bloat that I've seen, but currently there's no way to check for it except open the package and look at it.

This app is so helpful! Thanks a lot.

Well, I happen to have the very same question, and that would be a topic, OF this tool, as we are in fact, discussing the outcome, of its use, thank you.I'm also wondering why certain files appear to conflict, such as 2 different jewelry items, that cannot be worn, together. Does "conflict," mean that they simply can't be worn, simultaneously, or that they will cause issues, with your game, if they are even loaded, at the same time? People want to know this stuff; it's not necessary, to be snippy!

This tool already tells you why two items are conflicting: because they contain different resources with the same ID.

If that is not the sort of "conflict" you mean but you are having other issues with somebody else's CC, ask where you downloaded it.

Yes, that's the sort of conflict I mean, but it doesn't answer my question. Also, what's up with the "duplicates," that are actually recolors, NOT duplicates?

DISCLAIMER - I am a programmer, not a custom content designer, so some of the information below could be a bit inaccurate. If so, please let me know. It may help folks if an actual custom content designer is willing to write an expanded tutorial on using the conflict detector.

The result of the conflict depends on what the actual resource listed as a conflict is. The type of the resource is the first eight digits/letters of that resource ID that is listed, e.g. 015A1849 resources are the object model or geometry. If those are listed as a conflict then two of your custom content items have meshes with identical ID numbers but the actual meshes are different. Your game won't crash or anything (probably), but only one of those actual meshes would be used by the game. The differences between the two may be minor or major, there's no real way the program can analyze that, it just knows they are different - and depending on which one is actually used by the game (the last one loaded is the one which actually is used) BOTH clothing items will appear differently on Sim's bodies.

So, say that two hair styles have identical resource IDs that are internally different - one may have an odd bump due to a vertex being out of place that is fixed in the other hair style's mesh. If the "good" one is loaded last, all will look fine - but if the bad one is loaded last then both hair styles when used in the game will have that odd bump.

A duplicate means that the resouce ID numbers are identical AND the actual content is identical, so recolors may have the same type 034AEECB (CAS Part) information but the actual textures are different. One texture overrides the other and that is likely intentional for a recolor. The duplicate CASP information doesn't hurt anything - it is just loaded over and over again by the game, so if there's a lot of duplicates it could add substantially to the package loading time.

AGREED!

No, this would not really be the place for any in-depth explanations, I agree again.

Both of these are certainly possible! The latter idea would be more time consuming to code as I may (read probably) want to redo how I'm indexing things in a major way to try and keep the scan time efficient, but if you are correct that it's a major cause of package bloat it would be doable. Both will increase the scan time, and again, the second idea would increase it substantially particularly if the original game packages are included in that. I would essentially have to unpack and compute the hash for every resource in the game packages.

I thought I was done though!

When two CASPs have the same ID but are using different textures, wouldn't they be listed as Conflicting rather than Duplicate? The reference to the diffuse texture is in the CASP, so in order to reference a different texture they would need to be different internally.

Also, identical CASP IDs are very much not intentional for recolours. Only when overriding a CAS part (say you don't like a red Maxis shirt, change the texture to pink, and also change the colour tag from Red to Pink in the CASP) you'd use the same ID, or perhaps when you create multiple versions of the same thing for people to choose from. When people find identical CASP/COBJ/CWAL/ other catalogue item IDs in content that isn't supposed to override anything, they need to notify the creator and ask them to fix that.

LOL, sorry XD

I think it might not be super necessary to go though all the game files to find bloaty duplicates though -- comparing *between* CC items might already be quite useful, since when two CC items contain, say, an identical bumpmap texture with a different ID, the chance is quite high that those are *both* duplicates of the same Maxis texture (look at your own Packing Crate for an example -- isn't that the original, also the specular? That is what usually happens when people use S4S to create objects .. and the bump/specular maps are pretty large since they are always DXT5).

Of course it is also possible that a creator has multiple versions/recolours of the same item and each one contains the same custom bumpmap texture, but that should be relatively easy to tell by looking at the texture (and even in that case, it would be more efficient to use the SAME id .. no need to make the game load the same thing umpty-ten times under a different name).

(I actually discovered a tiny placeholder bumpmap in one of my own uploads that way -- was a copy of one of the EA blank bumpmaps that I had thrown in there because I meant to change/customise it later, then deemed that unnecessary and promptly forgot to remove that file (which still had the original ID). Some other creator obviously did the same thing, and now our packages showed up as "conflicting" (I believe -- might have been "duplicate" too) in your tool =).

Sorry, that is correct - I was thinking I might need to rephrase that to make it more clear but decided it was clear. It wasn't

The situation I was thinking of would I guess be called a "standalone" recolor of someone else's item. The CASP needs to be included in case the item that is being recolored is not installed, but the recolorer (if that's a word) rather than creating a new texture just edits the existing texture and drops it back in with the same ID. So the CASP contains a pointer to the same texture, but that texture resource is actually different.

So the program would see that as a duplicate CASP, but the DDS images would indeed conflict and would show up on the conflict tab as such.

So... not clear, and possibly a bad example - like I said, I'm a programmer not a content designer!

Lol - guilty! Did I mention I'm not a CC designer?

But it's not really guilt, I've said that S4S is a good program and in skilled hands folks can do great stuff with it (and I credit them on that mod, so it's no guilty secret that I used it). But the side effect of its ease of use is that it does allow people like myself, who probably should NOT be designing objects due to my lack of knowledge and skill, to end up with bloat like that.

So I can definitely see the advantage to scanning the whole installed game's packages for duplication. I'm just saying it's not something I'm likely to attack soon. What I'd like to do with this program next is to add descriptions of what the resource types are. I'm thinking add the four character "tag" to the list, with ideally a hover tooltip to then describe what that tag means. The type to tag information I can find easly enough (I think I may have a link somewhere even, but I'm sure a Google search would find it as well) but I could definitely use some assistance from someone knowledgeable in such matters with the descriptive text for tooltips.

That's what I figured, thank you. As for game mods, that conflict, I imagine that could be detrimental to gameplay, as well. I'm just trying to weed out the "harmless," duplicates and conflicts, vs. those that matter. I'm NOT a programmer, so I don't know half of what there is to know, which is why I ASK.

Maybe this list is useful to start with? https://github.com/s4ptacle/Sims4To.../Extensions.txt

That is the type-to-four-character-signature mapping that sp4e is using -- should contain all types from ts4 base game (plus probably a bunch of stuff that is still from ts3 >.<) . Some of these have meaningful file extensions, the xml types are currently all ".xml" though (not .buff or whatever) but that is probably not particularly relevant for your purpose.


Actually, the wiki makes more sense, because comments: http://simswiki.info/wiki.php?title...PackedFileTypes

I've also edited that wiki page now with a few additions. If you think the comments/info is too sparse, maybe it would make sense to also update the wiki as well?

There is a separate list for the text types as well, here: https://github.com/s4ptacle/Sims4To...xtResources.txt -- that is mostly the XML that you know anyway, but also stuff like lot object lists.

Thanks plasticbox, those will be helpful lists!

I think it would be useful/easier to read if I merge the file duplicate and conflict list into just "Files with Problems". So, in the screenshot below, the violet items are duplicates and the items in red are conflicts. I think this makes more sense, particularly for the file view?

Any thoughts on this? Should I merge them like this, or keep them separate? If people like the merged view, should I also merge the lists sorted by resource ID?

IMO violet and red are too hard to tell apart. I'd use maybe blue-grey for the duplicates? Or simply keep them in black, that makes the red stuff stand out more.

I think it makes sense to list issues package by package -- but for people who are trying to solve a problem, it is also useful to be able to see just the conflicts. Is it difficult to add a "Show Conflicts Only" filter tickybox maybe?

Ah, I've only just seen this -- could it be that you are mixing up CASP and GEOM here? To create a recolour of someone else's "item" ( = mesh, I think you mean?) there is no technical reason you would need the other person's CASP at all. The CASP is their catalog entry that links to all the textures and mesh, not the mesh itself ( = when the mesh that your recolour needs is not installed, the CASP will not help you). You would definitely not include it in your own recolour, neither changed nor unchanged, and you would also not include a texture that is using an existing ID!

For CAS parts there are actually two types of recolour: a "typical recolour" would be one that only changes the diffuse texture (and maybe the specular and/or self shadow too -- anything but the bumpmap); for that you create a *new* CASP with your own individual ID, that links to the mesh and textures it needs (all of the ones you made would have your own custom IDs as well) and also contains stuff like colour tags and catalogue sorting etc.

An "advanced recolour" would be one that also edits the bumpmap (say when you remove or add buttons or pockets, not just change the fabric texture/colour): for that you need to include a copy of the mesh, because the bumpmap is linked from within the mesh (which nobody likes but that is how Maxis have set it up). Plus the actual bumpmap as well of course, and all of the other parts as above.


The scenario you describe above would be more like a (partial) override of a CC item -- you might perhaps want to do that if you download a mesh by a CC creator who often updates their stuff, and it has one colour that you really hate, so you want to replace it with yours but without having to do that again every time you redownload the item. Make your own version of texture 0x12z472568497ghjf890324, plunk it into an /Overrides folder that you load with higher priority than the rest of your CC, and voilà -- if you do that you might also override the CASP as well (say to change colour tags or whatever). But that is really the only reason I can imagine to ever override somebody else's CASP resource, and it's already rather exotic.