doomsday.out is a log of what Doomsday is doing at runtime. In it will be logged errors, warnings and other pertinent information such as the command line options used at startup.
When reporting bugs and/or problems of a technical nature, you are expected to supply this file.
If a fatal error occurs Doomsday will attempt to show you the contents of the log automatically. However, if a real crash occurs (e.g., the Windows debug report dialog appears) you will have to look for it in your runtime folder (you will not be shown the log).
Doomsday can tell you where the Doomsday.out file is located.