|
|
|
By Na Than Assh Antti (na_th_an)
As soon as a QB programmer aims for a big project, he or she will (most likely) be trapped in the deep and dark well of memory problems. Sometimes your program refuses too compile no matter how many modules you split it into, sometimes it will compile but will do nasty things when executed, sometimes it will throw a "out of memory" error, and I can keep it going forever.
This is not just a QB issue (although it is easier to get such kind of problems with QB than with another similar compiler), but a 16-bits MS-DOS environment issue. You have 640 Kb of conventional memory to work with is real mode (QB, Turbo C++, and many other old compilers work in real mode), where you have to fit static, dynamic and code data. Far too little.
Back in the days, EMS and then XMS were introduced to have more memory to work with. This solves the problem to a point, 'cause you can only store data on EMS/XMS. No way to pass a program counter to EMS or XMS, so no way to actually execute a program from that zone. Sometimes you need more code. Many people just resignate and split their projects into several EXEs, and have to invent all sorts of tricks to have the data available from one to the next.
To solve that, Overlays were introduced. Overlays are just a pack of modules
(OBJ files, to aid in understanding) that are kept on disk until they are needed.
Overlaid programs contain a main module which is always in memory. When a
function is called, a little control program checks if it is loaded or not.
If it is not, it loads the overlay it is in from disk and calls the function.
That way you can have much bigger programs.
Don't worry, you don't have to call special functions, or code in an odd way. I feared those things until I met Jonathan Simpson and he pointed me to the correct document. The version of LINK.EXE included in PDS 7.00 or later does the work for you. You only have to tell LINK which object modules are on which overlays. Sadly, this can't be done from the IDE as far as I know, but not everything has to be a piece of cake. You have to compile your BAS file to OBJs using BC.EXE, and then run LINK.EXE to link the OBJs. The syntax for LINK is very simple. For example, say that you have seven object files (OBJs) named A.OBJ, B.OBJ, ... I.OBJ, and you want to organize them in three overlays. You just have to type in (provided you have your OBJs already created):
| LINK A+(B+C)+(E+F)+G+(I) |
The above command will create an EXE file but it will be a special, overlaid file. On it, object modules A and G will make the main portion which is always loaded in memory. B and C will make a overlay, E and F will make another and I will make another one. Note how object modules enclosed in parentheses get together inside an overlay, and how modules that are not enclosed in parentheses will be in the main section.
The program entry point (main section of your code) must be in the main section of your EXE file (not in a overlay). In execution, if a function in B.OBJ is needed, the overlay containing B.OBJ (which also contains C.OBJ) is loaded from disk, and the function is called. Imagine than then you need a function placed in C.OBJ now: no prob, the overlay is already loaded and the function will be called. Now you need some stuff at F.OBJ: well, no prob, B+C is unloaded and E+F is loaded from disk, then the function is called.
As you see, you don't have to do anything. Overlays are loaded and unloaded automaticly,
so you don't have to change a line of code, just organize your OBJs intelligently on
overlays... See next point.
You probably are thinking on this already: loading and unloading (specially the former) overlays takes time. Yeah, it does. And a lot of time compared to the time that is spent to call a function. You won't notice it when the load is outside a loop, but if it is inside a critical loop you will.
Overlays were a "patch solution" to code size problems. Like every other patch solution, it has lots of downsides. But luckily, these downsides can be ignored if we do things intelligently.
Most times, your program can be "sequentialized", that is, you can see how not all the SUBs and FUNCTIONs are used always, but that there are some kind of "stages". For example, in a game, you have a menu which calls to "options", "play" and "instructions". The "Options" SUB calls its own SUBs and FUNCTIONs that have nothing to do with those functions called by the "play" SUB, same for the "instructions" SUB. I mean that you, for example, are calling "DoScroll" from your "play" SUB constantly, but this SUB won't be ever called from the "options" SUB. Now you have a hint on how to organize your stuff: you can have an overlay with the menu system, another with the options stuff, another with the game itself and another one with the instructions.
"Hey, stop it a while", you may say. Yeah, I know, all those SUBs and FUNCTIONs that we have taken appart make use of the soundsystem and the graphics library. Easy: put the most commonly used stuff in your main section. That way they will always be loaded. Imagine how killer it were to place the soundsystem in an overlay and the game in another: everytime that a sound had to be played from the game module, it would have to be unloaded and the sound overlay would have to be loaded... No way :P
Another good division can be made in RPG games: Stuff your map engine in an overlay (along with the scripting engine and everything) and the battle engine in another. I've seen so many games that use different EXEs for the map engine and the battle engine. I always found that dirty. Now this is no longer needed :) and you have all your variables still on memory!
As you see, if you organize modules intelligently, the overlays system has no drawbacks.
In most cases, even if you don't need overlaying 'cause your code is small enough, it can
be better to overlay some parts of your program that are not used in critical loops so
you are saving memory for data in conventional memory. Plus, with modern computers and
operating systems, the time needed to load the overlay from disk is unnoticeable. Furthermore,
if you are using Windows all disk accesses are cached so most likely your overlay to be
loaded is somewhere in memory, so loading it is even faster. There is an even better
feature: if the EMS driver is loaded, overlays are copied to EMS as well when they are
loaded, and then they are retrieved from memory next time they need to be used.
Yeah, that's a bummer, but as I said before: nothing is perfect.
Each overlay cannot be larger than 256 Kb. There is also a maximum on how many overlays can be used: 64. That makes 64*256Kb = 16 Mb. I think you have fair enough with 16 Mb for your EXE, don't you?
When you are LINKing, an overlay cannot be specified as the first parameter. The first OBJ listed must be one of those included in the main section.
Each mode has to be compiled (with BC.EXE) using the same (or compatible) command line switches, or you'll screw everything.
Most EXE compressors screw up with QB overlays. Don't use them.
Create a .BAT file to compile and link automaticly. Study the LINK.EXE syntax so you can use .LIB libraries and pass parameters (in the above command LINK is only passed info about which OBJs will be linked, so it will prompt interactively for the missing info; you have to add these parameters to have LINK.EXE working automaticly).
Thanks to Jonathan Simpson for pointing me in the correct direction, concretely to BotTheMad's tutorial at Tek-Tips Forums.