Here you read what's storming in my brain...
Some may not appreciate it however I eventually decided not to continue the free version of Synalyze It!
This means that 1.4 was the last version offered for $0.
Let me explain why I chose to do this step. About 4 years ago when I started developing Synalyze It! I had no plan if, how or when to sell it in the future. I spent 100's of hours to make it the application I wished someone else would have written.
Several users called me crazy that I gave this piece of software away for free. In January 2011 the Mac App Store was launched and I took the chance to sell the more powerful Pro version while the other one remained free.
Later I gave the non-Pro version a price tag of $4.99 which is still a reasonable price however it never felt completely right because there was not much difference to the free version on this web site.
Stripping down the free version to a regular hex editor without the grammar features also wouldn't make much sense after offering all features and updates for a long time for free.
Shapefile is a very popular file format for storing geospatial vector data. To be precise, shapefiles consist of three mandatory file formats. One of them is the popular dBASE format which was very common in the MS-DOS times. (You remember? ;-)
There are now three new grammars available:
- .shp - the main shape files with the vector data
- .shx - contains an index for the shape file data
- .dbf - the dBASE file with attributes of the shapes
Although the .shp and .shx formats are structured quite simple, they have an uncommon mixture of little and big endian data, even in the main file header. The rest of the file is stored in little endian byte order.
The side effect of having grammars for shape files is the ability to decode dBASE files, at least on record level.
Here you find free shapefiles of many countries:
If you're interested in more details of the shapefile format, please have a look at the specification.
If you are someone who likes looking behind the scenes you probably did reverse engineer things already when you were a child. Always if you try to find out how things work - be it a radio, a computer program or a car, you apply reverse engineering.
There are several definitions and explanations available, Wikipedia has an own article on the topic. Originally the term was used in mechanical engineering but Elliot J. Chikofsky and James H. Cross II applied it to software engineering.
While the license terms of software products often do not allow to analyze the binaries you find many cases where reverse engineering is not only legal but also fun and useful.
There are several specialized tools available that support you in understanding binary files, both executable and data. One of these tools on OS X is Hopper disassembler that provides deep insight to programs even if the source code is not available.
Apart from executable files there are thousands of different file formats out there which mostly can only be read by the software that produced them. Synalyze It! was developed mainly to support the process of reverse engineering of binary files and give easy access to the contents once the format is described in a "grammar".
Not long ago a man contacted me and asked for help.
He had interviewed someone (audio+video), however the audio recording was interrupted at the end. Since the file was not readable by QuickTime and the interview couldn't be repeated you can imagine the situation in which he was.
Now, how to rescue a WAV file that wasn't written until the end?
Unfortunately the user didn't know anything about binary files, their formats and what could be wrong with an audio file. His first idea was to copy the end of a valid WAV file to the end of his interview to make it work.
So he sent me a short audio recording of 3 seconds to extract header and footer as a first step to repair his main interview file.
But… while WAV files indeed have a well-defined file header, there is no such footer - this means, the idea to fix the file end couldn't work! We had to take a different approach…
He then downloaded the free version of Synalyze It! and applied the grammar to his file. Because all the structures and their elements was Greek to him he proposed to send me the file.
In the previous article I explained how to create a PDF file from DocBook XML. One big advantage of using XML is that you can easily connect it to other sources.
To integrate documentation produced by Doxygen from your source files tell Doxygen to produce XML output (get it via MacPorts - "port install doxygen"):
where GENERATE_XML = YES is set in your doxygen.config.
/opt/local/bin/xsltproc --output DoxygenCompound.xml combine.xslt index.xml
In order to use the single Doxygen XML file (DoxygenCompound.xml) you need to translate it to a valid DocBook file:
/opt/local/bin/xsltproc --output DocBookChapter.xml Doxygen2Docbook.xsl DoxygenCompound.xml
The Doxygen2Docbook.xsl was written by me and probably needs to be adapted to match your needs. Finally reference the resulting DocBookChapter.xml in your main DocBook file:
Thousands of people use Synalyze It! now, many of them every day to work on their binary files. Although there is an online help available, a more comprehensive manual was missing yet.
There are many ways how to create technical documentation. If your primary goal is to produce a PDF file, most word processors are well-suited for the job. However, if you want to target different formats and incorporate other input data, without any doubt DocBook is the best choice.
DocBook is an XML language that allows to tag almost everything that can occur in technical documents. As every good XML language it doesn't mix structure and presentation. So you describe for example a command (<command>ls</command>) as such or a keystroke Cmd-k (<keycombo><keycap>Cmd</keycap><keycap>K</keycap></keycombo>), there's no information how to display them.
The actual presentation is added later in the process so you can be sure it's applied consistently to the whole document. To produce PDF you usually produce an intermediate XSL-FO file that can be processed by formatters like FOP or any of the commercial ones.
Some months have passed since version 1.2 has been published. Many people sent many good ideas and I had to select which features will make it into Synalyze It! 1.3.
Partially Apple required some changes, for example sandboxing. But even for the non-Pro version some things are added like optionally showing of hidden files or package contents in the file open dialog or a Spotlight importer for grammar files.
Users of the Pro version will be delighted by Python scripting that allows to write custom data types (that even translate changes back to the file!), work on grammars or files displayed in the hex view. If you want to get an impression what will be possible, have a look at The Script Page.
As always: Any feedback is appreciated!
ICC (color) profiles can be decoded now using the grammar you find on this page as well as a basic grammar for Audible (.aa) files.
A grammar for ELF binary files is on the way but needs some extensions in Synalyze It! which will be implemented in the next days.
Thanks to all users sharing their grammars for free!
Please consider providing your grammars also if they might be useful for others :)
All users noticing the bug when copying number elements with min/max values please download SynalyzeIt_220.127.116.11.zip
Enjoy your day
Enjoy & have a nice week :)
The next version of Synalyze It! is almost done... I hope.
Please have a look at SynalyzeIt_18.104.22.168.zip and report any bugs or whatever you think.
The version will come along with some new grammars for mach-o files and others :)
Have a great weekend!
Some days ago I installed Xcode 3.2.6 and unfortunately I built version 1.0.2 with it. Today a user noticed that the new version was not a complete universal binary anymore - the PPC part was missing.
Now the version is again a full universal binary - sorry for the confusion.
It seems that Apple tries to get rid of PPC quickly - with Lion even Rosetta will be dismissed...
Have a good <whatever you want>
Eventually Synalyze It! was released now also in the Mac App Store. Unfortunately the versions you find in the MAS and here on this site are not the same.
First, I had to remove some features due to MAS rules (synalyze shell tool and automatic online update as well as the automatic suggestion of grammars available on this site). Additionally in the version you get here are some fixes made while the version in the MAS waited for approval more than two weeks.
Hopefully I can keep both versions better on the same state in the future.
Thanks again for all the positive feedback, version 1.0.3 is already in the works :)
While waiting for the approval of Synalyze It! in the Mac App Store I created a grammar for Windows EMF files.
The version in the Mac App Store will have no automatic online update check and also the downloading of grammars has to be done manually due to the Mac App Store rules.
As soon as the version 1.0.2 will be approved I'll also provide it on this site.
All the best...
Since je ne parle pas francais very well I don't actually know what's been spoken in this french podcast of CocoaCast - could someone tell me more?
Localization of software can be really annoying so I wrote a Ruby script now to compare two UTF-16 encoded Strings files and a Rakefile that creates the German XIBs from the English ones using the strings.
The script is not yet really nice because of the redundant code but it does its job and tells me about missing strings in the localized version of the strings file.
Version 1.0.1 was just released and includes many enhancements and fixes. Please report any problems you find :)
Thanks for all the positive feedback! :-D
A lot has been improved and fixed in the past days. The most frequently asked extension - use a number element as repeat count of structures, has been implemented.
Please have a look before I release the next version and give me feedback.
Just in time I released today version 1.0. Bon anniversaire, Synalyze It!
There's much to come in the next year. Yesterday I worked on a grammar for ICC profiles and noticed that it works quite well except for two things: The repeat count of structures or structure elements must depend on other elements; even more difficult: There's first the reference (file offset), then the length of the referenced structure. Let's see how this can be solved ...
An interesting year has come to its end. Another lesson in accepting the world as it is, understanding the past, heading forward and focusing on the presence.Happy new Year 2011
С новым годом 2011
Quite some time has passed since the last beta version was published. Fortunately this doesn't mean nothing happens in Synalyze It! :)
My plan is to release a version 1.0 by the end of this year that is stable and has some additional features not seen in previous versions.
If you have a look at http://www.synalysis.net/formats.xml you may notice that the page looks the same as the old grammar page but since it's XML plus a XSL style sheet it can be easily parsed by Synalyze It!.
This means you can get suggestions for available grammars when you open some file and install the grammar on your computer with just one click. Could it be easier than this?
Of course this will work also for grammars installed locally.
In order to collect more grammars there'll be also a publish button that allows you to send a grammar file via email (hopefully to me :).
There are some other features users asked for I'll try to implement for the first non-beta version.
Happy Christmas (or whatever you like)
The hex editor part of Synalyze It! is now more or less complete - just a few things are missing.
So in the next weeks I'll focus more on the grammar and what can be expressed with it. Most importantly the length of string and binary structure elements must be able to depend on the value of number structure elements.
The new offset structure element type allows to display related contents that are spread over a file to be displayed next to each other.
Yes, I know... how to create a grammar is not really self-explaining. More documentation will come... :)
Not to forget: Thanks for all the feedback I got already! It helps to implement what's really needed and to create an application that makes Windows users jealous. :-D
P.S.: I found another (free) hex editor with additional features for the Mac: PeekIt.
When I started developing on the Mac, I was of course in the need of a capable hex editor. Even now there is no such comparable with what can be found on Windows. In the English Wikipedia, there is a useful Comparison of Hex Editors.
Since ordinary hex editing is not what I was looking for, I concentrated on the hex editors with "file structure view" (see second table on the Wikipedia page). If you're searching for a tool that disassembles machine code, this page is probably not for you while some of the applications do support this.
When I digged deeper and checked what the applications are really able to accomplish, only these products remained in my list (all on Windows only):
Additionally should be mentioned DataWorkshop which has some nice ideas but is completely outdated - not able to open and process large files and not really intuitive to use.
On the Mac there are only PeekIt and iBored offering more than simple binary file editing, the others (Hex Editor, HexEdit, hexedit, 0xED, Bless, beye, Hex Fiend, MadEdit, wxHex Editor) all don't help you much to understand what's encoded in a binary file. For PeekIt you can write a plugin for file analysis if you're a developer. The good thing all Mac application have in common: they're free :)