"MSVC" Hercules Build Instructions




Introduction


This document provides instructions and hints on how to build the "MSVC" version of Hercules (one that thus doesn't need Cygwin to be installed first in order to run).

Basically there are two different ways to do it.   Either:


  1. the Visual C++ Toolkit 2003 method   (if you're familar/comfortable with command-line tools),

    -OR-

  2. the Visual C++ 2005 Express method   (if you're more comfortable with GUI Windows development tools)   (recommended)


Microsoft's "Visual C++ 2005 Express" product is the free C++ version of their latest Visual Studio product offering. It's what I personally use for Hercules development. Effective April 19th, 2006, all Visual Studio 2005 Express Editions are free permanently.

Your other choice is the "Visual C++ Toolkit 2003" method (which is the same method we used before Microsoft came out with their free Visual C++ 2005 Express product (myself inluded)). With this method you open a Command-Prompt window and manually type in a command in order to build Hercules (as opposed to simply clicking a few buttons like you do when using the VC++ Express method).

The GUI method (using their "Visual C++ 2005 Express" product) is, in my opinion, the easiest of the two methods and is the one I personally recommend using. It involves simply installing their Visual C++ 2005 Express product as-is followed by their Platform SDK product afterwards.

The other method (the "Visual C++ Toolkit 2003" method) also involves installing their Platform SDK product, but in addition to that, requires a slightly more complicated temporary installation of their .NET Framework Software Development Kit (SDK) product in order to acquire a critical link library (which is provided/installed by default with their Visual C++ 2005 Express product, but which is not provided at all with their Visual C++ Toolkit 2003 product (which is why you have to temporarily install their .NET Framework Software Development Kit (SDK) product so you can acquire it), which is why I feel the VC++ Express method is easier).

Which of the two methods you choose to use is entirely up to you. Both of them work just fine and both are explained in further detail below.





Setting up the Hercules build environment
(Summary)


The overall SETUP process is basically very simple (but a slight pain-in-the-butt for the Toolkit 2003 method) and is summarized below.

Detailed instructions follow further below following the below summary.


Note: once you've setup your build environment, (which you only need to do once and never again) the process of actually building Hercules is extremely simple: just open a Command-Prompt window and enter the proper command!  That's it!  Or, if you decide to use the Visual C++ 2005 Express Edition product instead, it's as simple as opening our pre-provided "Hercules.sln" Solution file and then clicking the 'Rebuild Solution' button!)





Setting up your Build Environment
(Visual Toolkit 2003 method)

SUMMARY


  1. Download and install the Visual C++ Toolkit 2003   (to get their compiler)

  2. Download and install the Platform SDK   (to get their headers)

    Note: you shouldn't need the full SDK. You really only need the basic Windows stuff (to get the current set of Windows headers), but if you're unsure as to what you actually need and what you don't, then it shouldn't hurt to go ahead and install the whole thing.

  3. Download and TEMPORARILY install the .NET SDK   (to get the MSVC LIB & DLL)

    1. Download and install it
    2. Copy the needed MSVCRT.LIB file to the Platform SDK "libs" directory
    3. Copy the needed MSVCR71.DLL file to someplace where Windows can find it
      (if you don't already have it on your system somewhere)
    4. UNINSTALL it

  4. (optional)  Download and "install" the ZLIB package   (to get the DLL and headers)

    1. Download both the binary AND the source packages
    2. Unzip to a temporary directory
    3. Create your own slightly more permament directory for them somewhere
    4. Copy the needed files to their new [semi-] permanent home (so the Hercules build process can find them there)
    5. Delete your temporary directories
    6. Define a ZLIB_DIR environment variable pointing to your directory so the Hercules build process can find the needed files.

  5. (optional)  Download and build the BZIP2 package from Cygwin   (to get the LIBBZ2.DLL and headers)

    1. Download the bzip2 source package
    2. Unzip to a temporary directory
    3. Fix their makefile
    4. Run the makefile (to build the DLL)
    5. Copy the resulting DLL and associated header file to a more permanent directory somewhere
    6. Define a BZIP2_DIR environment variable pointing to your directory so the Hercules build process can find the needed files.

  6. (optional)  Download and "install" the Perl-Compatible Regular Expressions (PCRE) package   (to get the DLLs and headers)

    1. Download the PCRE Binaries and Developer files packages
    2. Unzip each into a temporary directory
    3. Create a directory to be the permanent "home" for the package
    4. Create a 'bin', 'include' and 'lib' subdirectory beneath it
    5. Copy the necessary files into those subdirectories
    6. Define a PCRE_DIR environment variable pointing to the permanent home directory for the package.

  7. Create a cmd file ("batch" file) to initialize and invoke the build process!   (to create your own Hercules!)





Setting up your Windows Build Environment
(Visual C++ 2005 Express method)

SUMMARY


  1. Download and install Visual C++ 2005 Express.

  2. Download and install the Platform SDK   (to ensure you have a complete set of the latest Windows headers)

    Note: you shouldn't need the full SDK. You really only need the basic Windows stuff (to get the current set of Windows headers), but if you're unsure as to what you actually need and what you don't, then it shouldn't hurt to go ahead and install the whole thing.

  3. Add the Platform SDK directories to your Visual C++ 2005 Express build directories list


    Note: the above three steps are actually covered in more detail on Mirosoft's own "Using Visual C++ 2005 Express Edition with the Microsoft Platform SDK" web page.


  4. (optional)  Download and "install" the ZLIB package   (to get the DLL and headers)

    1. Download both the binary AND the source packages
    2. Unzip to a temporary directory
    3. Create your own slightly more permament directory for them somewhere
    4. Copy the needed files to their new [semi-] permanent home (so the Hercules build process can find them there)
    5. Delete your temporary directories
    6. Define a ZLIB_DIR environment variable pointing to your directory so the Hercules build process can find the needed files.

  5. (optional)  Download and build the BZIP2 package from Cygwin   (to get the LIBBZ2.DLL and headers)

    1. Download the bzip2 source package
    2. Unzip to a temporary directory
    3. Fix their makefile
    4. Run the makefile (to build the DLL)
    5. Copy the resulting DLL and associated header file to a more permanent directory somewhere
    6. Define a BZIP2_DIR environment variable pointing to your directory so the Hercules build process can find the needed files.

  6. (optional)  Download and "install" the Perl-Compatible Regular Expressions (PCRE) package   (to get the DLLs and headers)

    1. Download the PCRE Binaries and Developer files packages
    2. Unzip each into a temporary directory
    3. Create a directory to be the permanent "home" for the package
    4. Create a 'bin', 'include' and 'lib' subdirectory beneath it
    5. Copy the necessary files into those subdirectories
    6. Define a PCRE_DIR environment variable pointing to the permanent home directory for the package.

  7. Start Visual C++ 2005 Express, open our pre-provided "Hercules.sln" Solution file and click on the 'Rebuild Solution' button.





Download the C++ Toolkit Compiler and Platform SDK

( Visual Toolkit 2003 method -- detail )


The Microsoft Visual C++ Toolkit 2003 includes the command line versions of the Visual C/C++ compiler and linker, as well as the standard C/C++ Library.

Download the installation file (a 32MB download) and run it to install the Toolkit. The resulting   C:\Program Files\Microsoft Visual C++ Toolkit 2003   directory should only consume about 30MB of disk space. Once installed, run the 'cl' program at least once (from the Visual C++ Toolkit Command Prompt window) before installing the Platform SDK.


The Platform SDK Web Install contains the include files and libraries needed to create applications for Win95/98/ME/NT/2000/XP.

Important: Before installing the Platform SDK, you should first launch "Visual C++ Toolkit 2003 Command Prompt" and run the 'cl' program at least once (by entering the command   cl   inside the Toolkit Command Prompt window) in order to register the VC++ environment variables properly, and then verify that the environment variable "VCToolkitInstallDir" exists and is set correctly.

Also note:  you may need to add "download.microsoft.com" to the Internet Explorer "Trusted sites" zone before you can install the SDK (depending on your current security settings).


Obtain the MSVCRT.LIB and other files

( Visual Toolkit 2003 method -- detail )


To build the DLL version of Hercules then you will need the Microsoft file MSVCRT.LIB which is not included in the free Visual C++ Toolkit 2003. You can however obtain this file by downloading and installing the .NET Framework Version 1.1 Software Development Kit (SDK). (Warning! This is a 108MB download and may take a while on a slow connection)

After installing the .NET Framework SDK, proceed with the following steps:


  1. After installation, copy the file msvcrt.lib from the directory:
    C:\Program Files\Microsoft Visual Studio .NET 2003\Vc7\lib
    to the directory:
    C:\Program Files\Microsoft Visual C++ Toolkit 2003\lib



  2. Then copy the file NMAKE.EXE from:
    C:\Program Files\Microsoft.NET\v1.1\Bin
    to:
    C:\Program Files\Microsoft Visual C++ Toolkit 2003\bin



  3. Then copy the file SetEnv.cmd from:
    C:\Program Files\Microsoft Platform SDK
    to:
    C:\Program Files\Microsoft Visual C++ Toolkit 2003



  4. Then copy the file cvtres.exe from:
    C:\WINNT\Microsoft.NET\Framework\v1.1.4322
    to:
    C:\Program Files\Microsoft Visual C++ Toolkit 2003\bin



Once the above files have been copied, you can then UNINSTALL the .NET v1.1 Framework SDK (via the Control Panel "Add/Remove Programs" applet) to free up about 250MB of disk space. (Note! Be careful! Do not uninstall the Platform SDK or the Toolkit! Only uninstall the .NET v1.1 SDK and nothing else!) (and delete that 108MB installation file you downloaded too; it's no longer needed either)





Setting up ZLIB Support

( common to BOTH methods -- detail )


ZLIB is a compression algorithm written by Jean Loup Gailly and Mark Adler and may be used in the Hercules project pursuant to the ZLIB License (a copy of which may be seen at http://www.zlib.net/zlib_license.html).

In source form, the Hercules project does not contain any ZLIB source code at all.

In binary form however, the Hercules project may include an unmodified version of the ZLIB runtime DLL in addition to its own distribution binaries.

The "ZLIB_DIR" environment variable defines the location of where the needed files are for building a version of Hercules that supports ZLIB compression. The makefile used by the Hercules build process tests whether this environment variable is defined or not in order to know whether you wish to build ZLIB compression support into Hercules or not.

Note: you can define an environment variable either temporarily via the 'set' command or permanently via the Control Panel 'System' applet. Refer to Windows 'Help' information for: "environment variables, adding" for more information.

If 'ZLIB_DIR' is undefined when you build Herc, then an attempt is made to locate the ZLIB library in the directory:   winbuild\zlib\win32_32.   If ZLIB_DIR contains a bad path, nmake exits with an error unless the ZLIB_DIR variable is set to the special keyword "NONE" (in which case support for ZLIB compression is obviously not generated). So the choice is yours: you can either leave it undefined -or- define it to "NONE" if you don't want ZLIB support. Otherwise if it's defined to anything other than "NONE", then it must be defined to a valid path.

ZLIB_DIR should contain the *BASE* path for the ZLIB files. This means that the following files (and directories) are expected to exist in the specified ZLIB_DIR directory:

Remember to "nmake clean" if the build type is changed, the ZLIB location modified, etc..

NOTE: The  zlib1.dll  Dynamic Link Library (DLL) will be copied to the appropriate build target directory as part of the build process in order to ensure that the DLL is obtainable by the platform PE(*) loader at run time.

(*) PE stands for Program Executable and designates Windows .EXE & .DLL executable file formats.


Obtaining 'ZLIB'

Building from the distributed Hercules source tree does not by default incorporate ZLIB as a compression mechanism, since:

.. Therefore ..

In order for Hercules to be built with ZLIB support (DASD, TAPE), go to:  http://www.zlib.net  and locate the download for the 'zlib compiled DLL'. (Note: This is only a 75KB download.)

Extract that .zip file to the   winbuild\zlib\win32_32   directory, relative to the Hercules source-code directory   -OR-   instead, you may extract the file to your alternate location and then set the ZLIB_DIR environment variable accordingly (as explained further above).

The Hercules project is currently known to succesfully build with version 1.2.2 (current at the time of writing) of the ZLIB compression library.





Setting up BZIP2 Support

( common to BOTH methods -- detail )


BZIP2 is a freely available (open-source (BSD-style) license), patent free (as far as the author knows), high-quality data compressor written by Julian R Seward. It typically compresses files to within 10% to 15% of the best available techniques (the PPM family of statistical compressors), whilst being around twice as fast at compression and six times faster at decompression.

In source form, the Hercules project does not contain any BZIP2 source code at all.

In binary form however, the Hercules project may include an unmodified version of the BZIP2 runtime DLL in addition to its own distribution binaries.

The BZIP2_DIR environment variable defines the location of the BZIP2 link library and header file. If the BZIP2_DIR environment variable is undefined, then BZIP2 support will not be generated when you build Herc. Furthermore, if BZIP2_DIR defines a bad path, nmake will exit with an error, EXCEPT if BZIP2_DIR is set to the special keyword "NONE" (which, as far as the Herc makefile is concerned, is the same thing as it not being defined at all).

Note: you can define an environment variable either temporarily via the 'set' command or permanently via the Control Panel 'System' applet. Refer to Windows 'Help' information for: "environment variables, adding" for more information.

BZIP2_DIR should define the *BASE* path for BZIP2. That means that the following files are expected:

Remember to "nmake clean" if the build type is changed, the BZIP2_DIR location modified, etc..

Note: The   libbz2.dll   Dynamic Link Library (DLL) will be copied to the appropriate build target directory as part of the build process in order to ensure that the DLL is obtainable by the platform PE(*) loader at run time.

(*) PE stands for Program Executable and designates Windows .EXE & .DLL executable file formats.


Obtaining BZIP2

Building from the distributed Hercules source tree does not by default incorporate BZIP2 as a compression mechanism, since

.. Therefore ..

In order for Hercules to be built with BZIP2 support (DASD, TAPE), go to  http://www.bzip.org  and click on the Downloads link, and locate the link entitled: "Here is the 1.0.3 source tarball, which includes full documentation" near the beginning of the page.

Yep, that's right! You're going to have to manually build the BZIP2 dll for yourself from source!   (either that or obtain a pre-built copy of it (and associated header/lib files) from someone else you trust)

Note: This is a 650KB download, and the file is in tar.gz format, so you'll need WinZip (or some other utility that understands tar.gz format) in order to unzip it.

Extract the files to a directory of your choosing and build the DLL using a modified version of the supplied makefile.msc:



Once the above changes to the makefile have been made, then open a "Visual C++ Toolkit 2003 Command Prompt" window (it should be in your Start menu somewhere), and enter the following commands to build the DLL:


    setenv
    cd \
    cd xxxxxx...  (i.e. to wherever your bzip2 source files are)
    nmake -f makefile.msc


Once it's built, then copy the resulting DLL, lib, and bzip2.h header file to your defined BZIP2_DIR location and then build Herc.





Setting up PCRE Support
(Perl-Compatible Regular Expressions)

( common to BOTH methods -- detail )


PCRE (Perl-Compatible Regular Expressions) is: "a set of functions that implement regular expression pattern matching using the same syntax and semantics as Perl 5. PCRE has its own native API, as well as a set of wrapper functions that correspond to the POSIX regular expression API. The PCRE library is free, even for building commercial software."

In source form, the Hercules project does not contain any PCRE source code at all.

In binary form however, the Hercules project may include an unmodified version of the PCRE runtime DLLs in addition to its own distribution binaries.

NOTE: the Perl-Compatible Regular Expressions library is needed only to support the Hercules Automatic Operator (HAO) Facility. If you do not plan to ever use the Hercules Automatic Operator facility, then you do not need to install PCRE support and may safely skip this step.

The [slightly more] detailed installation instructions for PCRE are as follows:


PCRE Installation Detail


  1. Download the Binaries and Developer files packages from the PCRE web site.

  2. Unzip each into a temporary directory of your own choosing

  3. Create a directory named whatever you want, to be the permanent "home" for the package. Remember this name as it will be used in step 6.

  4. Create a 'bin', 'include' and 'lib' subdirectory beneath the above mentioned PCRE 'home' directory.

  5. Copy the following files from the previously mentioned temporary unzip folder into the above mentioned 'bin', 'include' and 'lib' subdirectories:

    • Copy the "pcre3.dll" and "pcreposix3.dll" DLLs from the "%temp%\pcre-6.4-1-bin\bin" directory into the PCRE permanent "home" directory's 'bin' subdirectory.

    • Copy the "pcre.h" and "pcreposix.h" header files from the "%temp%\pcre-6.4-1-lib\include" directory into the PCRE permanent "home" directory's 'include' subdirectory.

    • Copy the "pcre.lib" and "pcreposix.lib" link library files from the "%temp%\pcre-6.4-1-lib\lib" directory into the PCRE permanent "home" directory's 'lib' subdirectory.

  6. Define a PCRE_DIR environment variable pointing to the main permanent "home" directory for the package.

  7. (optional) Delete/discard the original temporary unzip directories unless you wish to hang onto whatever other files they may contain.





Add the Platform SDK directories to your
Visual C++ 2005 Express build directories list

( Visual C++ 2005 Express method -- detail )


Note: the below instructions are simply a more detailed version of the same instructions provided on Mirosoft's "Using Visual C++ 2005 Express Edition with the Microsoft Platform SDK" web page.


In this step we tell Visual C++ 2005 Express where the Platform SDK "Executables", "Include" and "Library" directories are so that it can find various header and library (.LIB) files needed during the build process.

Select "Options" from the "Tools" menu. An 'Options' dialog should appear. (Duh!)

In the "Show directories for:" combo-box, select: "Executable files" (if it's not already selected), and then click on the yellow folder icon to create a new entry in the list. The new entry will be initially empty and will need to be filled in. To fill it in, click on the ellipsis button at the end of it and then browse to where you installed Platform SDK and select the 'bin' directory and click 'Open'. That should add that particular directory to the list. BUT... it may not be in the proper position, so we'll need to fix it.

Click on the up/down arrow to move the entry so that it comes first, before all others.

Then do the same thing again, but this time selecting the Platform SDK's 'bin/winnt' directory instead. Position it in the list so that it comes second, immediately following the 'bin' entry.

Done! ...

... with THAT directory; we still have two more to go however.   :)

Now click on the "Show directories for:" combo-box again and this time select "Include files". Add a new entry like before, but this time add Platform SDK\include as the new entry, and move it to it's proper position (i.e. first, before all others).

Lastly, add your Platform SDK\lib directory to the "Libraries files" list and position it too so that it comes first, before all others.

That's it!   You're done.   You're now ready to build Hercules.   :)




The Hercules Build Command

( Visual Toolkit 2003 method -- detail )


Open a Command-Prompt window, navigate to the directory where the Hercules source code is, and enter the following command:

    makefile.bat  RETAIL  makefile-dllmod.msvc  2  -a

The third argument ('2' in the above example) is the maximum number of CPU engines you wish your Hercules to support. There is no default; the argument MUST be specified on the makefile.bat command (although Hercules itself defaults to 2 CPU engines if the MAX_CPU_ENGINES variable is not specified at build time).

The last argument ('-a' in the above example) requests a "full" re-build of everything. You may optionally leave off this argument to build only what is necessary(*) or specify 'clean' instead to clean-out everything in your build directories.


(*) Note however that I have personally experienced unusual/unpredictable Hercules behavior at times whenever I fail to do a full build (i.e. a "rebuild all" with the '-a' option). It doesn't happen very often (only sporadically under as-yet undetermined specific conditions), but whenever it does happen, it's so very confusing as to why Hercules is behaving so weirdly / unusually that, IMO, it's probably safest to always do a full rebuild each time.(**)

(**) I suspect the cause may be related to the fact that we're specifiying link-time optimizations but then not re-linking (rebuilding) all of the modules, thereby confusing the Global Optimizer, but that's just a guess. <shrug>   Bottom line: I would highly recommend always doing a full (complete) rebuild of everything by specifying the '-a' option each time you build Hercules.




Building Hercules using the Visual Studio IDE

( Visual C++ 2005 Express method -- detail )


Default Visual Studio 7.0 and 8.0 (Visual C++ 2005 Express) solution and project files (as well as Visual Studio 6.0 workspace and project files) are included as part of the Hercules source-code distribution. Simply open one of them (i.e. either the Solution or Workspace file, depending on which version of Visual Studio you're using) and click your 'Build' or 'Rebuild All'(*) button.   That's it!     It truly is that simple!    :)

The default makefile project simply invokes "makefile.bat" which in turn simply invokes the 'nmake' command for "makefile-dllmod.msvc" (after calling a few key batch files to define the build environment).


(*) Note however that I have personally experienced unusual/unpredictable Hercules behavior at times whenever I fail to do a full build (i.e. a "rebuild all" with the '-a' option). It doesn't happen very often (only sporadically under as-yet undetermined specific conditions), but whenever it does happen, it's so very confusing as to why Hercules is behaving so weirdly / unusually that, IMO, it's probably safest to always do a full rebuild each time.(**)

(**) I suspect the cause may be related to the fact that we're specifiying link-time optimizations but then not re-linking (rebuilding) all of the modules, thereby confusing the Global Optimizer, but that's just a guess. <shrug>   Bottom line: I would highly recommend always doing a full (complete) rebuild of everything by specifying the '-a' option each time you build Hercules.





Well, that's pretty much it I guess.   <shrug>


If you have any questions or problems, feel free to ask the hercules-390 group for help.


Enjoy!   :)



  "Fish"  (David B. Trout)
    fishsoftdevlabs.com

"Programming today is a race between
software engineers striving to build bigger
and better idiot-proof programs, and the
Universe trying to produce bigger and better
idiots. So far, the Universe is winning."

- Rich Cook