Hints for WebStar ServerMasters
(Note: We are not running WebStar 5, so these hints don't apply.) WebStar 4.x should be the frontmost process for best server performance. We use PullProcess to pull WebSTAR to the front (just in case we forget to leave it there when we log off from Timbuktu). This allows WebSTAR to get a very high percentage of the processor's time. FileMaker Pro, especially with Web Companion enabled, is a processor hog. We've found that we can fine-tune processor time allocation by using Peek-a-Boo to give WebStar maximum processor time, leaving FileMaker at normal.
We have Rumpus set to be "well-behaved." Because we run a bunch of CGIs, we have WebStar's CGI/Server setting more toward CGI.
Following is Steve Crisp's list of recommendations for webmasters. His opinions are his own, but he has lots of good advice here, particularly if you're running WebStar & MGI. I'm pretty sure he won't mind my copying it here:
"This is long and it is late. I completely disclaim all spelling and grammar errors. Enjoy... Through years of experience with WebStar, I have found the following set up to be the most stable. I'm gonna go screen by screen in the WebStar admin and cover all applicable settings. Then I'm gonna have some fun.
- File Names: Server name: the canonical name of your machine or its IP number. You are using canonical names for all your machines, aren't you?
- Index: index.mgi if you are using MGI on every page or index.html if you are restricting the default home page to such. If the data cache is installed, you can put multiple home page names in the field, though I would not recommend that you use WebStar data cache.
- Pre Processor: keep it blank unless you have an overwhelming need to actually use a preprocessor. Unless you are still using that Clearway thing, there is no reason to use a preprocessor.
- Max Connections: it depends on the use of any given machine, but I keep it between 150 and 250 in order to handle peak and spike loads. In no case should you set it about 250 since WebStar will probably blow up at that level anyway.
- Performance: the slider goes to the far right for Better Server
- Caching: File Info Cache: off with the File Info Cache Size set to zero just in case
- Data Cache: the whole thing should be greyed out since the data cache should not be installed, but if it is then Disk Cache Size should be set to as high a figure as your RAM will accomodate. I mean, if you are going to actually use it, you may as well take full advantage of all you can get.
- Maximum File Size: entirely dependent on what you are serving. If you are serving primarily small images and html pages then 50K will be more than sufficient. If you are serving large PDF files and the like or downloadable software then many megs may be needed. On the other hand, if you ARE serving a lot of large files, you are much better off getting an old 7200 or something and dedicating that machine to serving only your large files.
- Check File Modification Dates: it all depends again on the use of your machine. If you are the only one on the machine, you can set it for as long as you like or even never since you have total control over the cache flush when you make an update. If you have a lot of clients on a single machine or if there is a whole bunch of FTP traffic where folks are making constant changes, then set it low like at four minutes. Remember though that checking the modification date is I/O intensive as well.
- Auto Flush Cache: another tricky one depending on machine use. If the files do not change very often then set it as high as you can or turn it off completely. If files change constantly, then set it low. Ten minutes seems to appease most folks, though some will get testy even when they have to wait that long for changes to appear.
- SSI:
- Default Character Set: check Latin 1 under almost all circumstances. I really can't think of why you would want to check the other two.
- Data Caching: off. When you are using MGI, you have no need for SSI at all
- #EXEC and #Include Comments: check Disable Commands or at worst restrict them to the CGI bin. NEVER allow unrestricted use of comments. EVER.
- Suffix Mapping: All the defaults hold true except the following: You need to add .mgi if you are using MGI If you are processing .html or .htm via MGI then you need to change the action from binary to MGI on those suffixes. Add the .zzz to fix the problem with WebStar getting into a loop. The action for .zzz is binary, the type is *, the creator is * and the mime type is text/html. exe should be processed via MGI and not binary. That will prevent your server from crashing, though it will also prevent you from serving out exe files. Although you should never serve out a raw .exe file anyway. You should always zip or tar it for download and as such it would have a zip or .tar suffix.
- Allow/Deny: Just leave it alone and blank. If you mess with it, you will get in trouble. Of course it has its uses, but we use MGI (and occasionally WebStar realms) for any restrictions rather than WebStar's allow/deny because of how complicated it is to set up properly.
- Logging Options: Here's a neat one. First set your Status Report Delay to 15 seconds. It really doesn't matter, but 15 seconds gives you a nice histogram that is representative of what is happening on the server. Now, where do you write the file to? We set up a WebStar folder completely outside the WebStar heirarchy which contains the WebStar admin app and the logs folder. Then we write the log to that log file and not one located inside the Webstar heirarchy itself. Why do we do that? If the log is outside the WebStar toot there is absolutely no way that anyone can get it even though it is protected by the 3-Omega creater code.
- Log Format: We use the WebStar Log Format as our default and the fields are in this order... Result, Method, Date, Time, URL, Bytes_sent, Referer, Hostname, Agent. That is simply my preference and gives me all the information I need. Plus it makes the WebStar window easier to read. By putting the Result in the first position I can easily see if I am getting errors (ERR)
- On the Connection Settings tab in the WebStar Admin: Make absolutely sure that Restrict CGIs to CGI bin is checked on. The only time you uncheck it is if you have a really good reason to do so. If you keep it unchecked, then anyone with FTP access to your machine can upload any CGI they choose and launch it. Not good. Also, leaving it unchecked in conjunction with file upload whether via WebStar or MGI allows ANYONE at all to potentially upload an executable file and launch it from within a client folder. Really not good. The other setting -- Use DNS for Server And Client IP Lookups -- I leave unchecked. Doing a DNS lookup on each transaction just to write it to the log file takes a whole lot of time. Let your log processing program do it for you off machine.
Now, my plugins folder... Here is what is in that folder:
- a folder called MGI Files The MGI 2 plugin itself the WebStar Admin plugin (not the actual admin application)
- The WebStar Admin Data folder WebStar FTP plugin for those using WebStar FTP
- the WebStar FTP Data folder for those using WebStar FTP
- WebStar PowerKey Pro Tickler for those using Power Key Pro
- WebStar SSI plugin even though you are not using it; it is required
- WebStar SSI setting which are automatically created
- Welcome Plugin for those using Welcome
- Welcome Plugin folder for those using Welcome
- If you are using WebStar Virtual Hosts you will have the Webstar Virtual Hosts plugin and nothing with Welcome.
- If you are using both MGI 1 and MGI 2 on the same machine, you will also have the MGI Server Plugin in addition to the MGI2 plugin.
- If you are still using WebStar 3.x for some reason, you will also have the appropriate Nitro plugin that suits your needs. ALWAYS run Nitro if you are using WebStar 3.x. NEVER use Nitro if you are using WebStar 4.x.
- Do not use WebStar 5 -- it sucks at least for now.
- And would you PLEASE just bite the bullet and get rid of WebStar Mail. It sucks worse than anything else. EIMS and Mailburst are inexpensive and can run very effectively on a 7200 or a stock 7500 -- both of which can be purchased for about 50 bucks now.
- Further, get rid of Proxy and all that other crap that installs with WebStar. None of it works properly.
- Use MGI for file upload; the WebStar file upload plugin sucks.
- FTP still has a memory leak, but for now I am stuck with it due to technical and political reasons. Anything else that installs with WebStar other than what is noted above sucks.
- Your WebStar memory settings should be set for at least 80 megs regardless of how few connections you have set. The more connections, the more memory you will need. Generally we have memory set at one meg per connection over 80 with 80 megs as the minimum. Thus, a machine that allows 200 connections will have a memory allocation of 200 megs. And if you have a lot of database processing going on, add another 50 megs for good measure. Remember, those are perhaps minimum values. You can never go wrong adding memory to an application. Just be careful, though. What you do not want to happen is to restart WebStar and find that your free system memory is less than your application allocation. It happens when you open and close apps on the server and the free memory gets segmented. Generally, you should have twice the memory you actually need installed on your machine if all the applications you often use are running at the same time. Yes, it's overkill, but it's better to be safe than sorry and with memory so cheap there is no reason not to.
- Speaking of memory management... ALWAYS Norton the machine on a periodic basis and not just checking files with Disk Doctor either. Run a Speed Disk anytime your fragmentation gets above 0.5 percent. And don't just defragment -- optimize. Nothing will slow doen a drive like fragmentation. On a four gig drive it takes about three hours. Just do it in the middle of the night every once in a while. By the way, in almost six years now we have not had a single problem when running Norton. Not one. Some folks report issues, but we have never seen them. Just keep Norton completely current with respect to the OS you are running and everything will be fine.
- Also keep your drivers updated on the hard drives.
- And always use the latest Apple Disk First Aid from time to time as well. Just make sure it is the latest version, though.
- Also, keep an eye on the MGI databases particularly the shopping basket database. If folks add things to the shopping basket databases and do not check out, those things stay in there unless you set up an autoflush (see the MGI docs.) If the databases grow then the machine slows dows as they get bigger and bigger. Also check the counter database from time to time especially if you change pages often and they all have counters. We had one domain that, over time, accumulated over 1200 counters -- most of which were no longer in use. Yet everytime a current counter was hit the whole database had to be processed. Needless to say it put a major strain on the server. Watch out for loops. We have built in protections with mgiLoop that prevents infinite loops, but you can still end up in trouble with multiple loops per page hit that are all doing a lot of looping. Looping takes up a lot of CPU time.
- Also watch out for a lot of file includes on a single page. File includes are very I/O intensive and can bog down a machine if grossly overused.
- Turn off Virtual memory. You do not need it. Buy physical RAM if you need more memory. VM works nicely when you are running Photoshop, but it will kill a server that is TCP/IP intensive. Just turn it off and be done with it once and for all.
- And while we're at the Memory Control Panel...keep your Disk Cache at the default setting. I have never seen any kind of performance change regardless of that that is set at. RAM Disk is off.
- Set Energy Saver completely off and then go under Advanced Settings. See that little check box for the power failure. Make sure it is checked on. You would be amazed how many folks do not turn it on then wonder why their machine does not reboot if the power goes off.
- File Sharing: off. Don't even think about turning on file sharing on a server.
- Monitors: go 256 colors on a middle size setting. Do not set it for millions of colors at 12 billion by 14 billion pixels or some such thing. Why waste processer time on a machine whose display quality doesn't matter.
- Quick Time Settings: you need QT installed, but make sure that both check boxes under Auto-Play are unchecked unless you want to get a virus.
- TCP/IP: Again...go to Options and uncheck that box that says Load Only When Needed.
- Date and Time: set your clock to indicate seconds especially if you are using TB2. It is easier to tell if that machine is stalled or crashed if the seconds stop ticking off. And make sure that your time synchronizer is set to manual. If it is set to check periodically and the time server is not available, it can hang the machine.
- General Controls: See that check box under Check Disk? It had better be unchecked.
- Software Update: I like software update, but always do it manually. Never automate it. EVER. Get rid of the Control Strip control panel and its extension. It has a conflict with TB2 that will crash your machine.
- And speaking of TB2...I place an alias of TB2 application into the startup folder so that the app (not the extension) launches on reboot. I have found that sometimes if the app is not already running when you try to TB2 in, the machine will hang and launching TB2 at startup does not harm anything.
- Any extension or control panel that you can not absolutely justify having installed -- take it out. That includes things like printer stuff, USB and FireWire if you are not using USB or FireWire. And always remove anything that allows you to play DVDs and CDs. Who listens to that stuff on a live server anyway?
Experiment. Get it down to the bare minimum so that the machine will still boot and serve properly. If you take out too much or the wrong thing, the machine will not boot or serve. If that happens, put it back in. If you use Software Update, watch out for the security oriented extensions; they have to be in there. Who needs the GL or sprockets stuff. Get ruthless.
How about backup strategy... Here's what I do: I have two internal drives, one is twice as big as the other. I used to use externals for this, but we have had problems with external fans where the drive overheats and blows up. Also use no more than 7200 RPM drives and try to use 5400 RPM drives. They run much cooler and last a lot longer. Never use a 10,000 RPM drive on a machine that never gets turned off. We have lost every single 10,000 RPM drive that has ever been hooked up to a server. You're not doing video production here; you don't need it. Drive one: Three partitions. Partition one- system software and other applications including that WebStar admin and log folder noted above. Partition two- WebStar root folder and aliases to client folders. Partition three- Client folders. Set the partition size to whatever is comfortable, but don't forget that under MGI2, the data files are stored in the WebStar root partition so give it a bunch of room to grow. We run 4 gig primary drives with the partitions set at 1024 megs for apps, 1024 megs for WebStar, and the balance for client files. That also gives plenty of room for the WebStar logs to grow. By the way, I recommend against archiving WebStar logs automatically. Some folks have had huge problems with doing so, though some folks have no problems at all. it is just as easy to turn off logging temporarily, rename and move the log, then turn back on logging which automatically recreates a log file. I usually pull any given log when it hits about 200 megs and move it over to the machine we have dedicated to stats. We currently have over four gigs of log files to process each month; your mileage may vary. Remember also that if your log file should fill up all the available space on the drive, WebStar will stop serving files even though the screen looks like it is serving files. Very bad. Give those logs lots of room to breath. Drive two: Six partitions in pairs equal in size to the partitions on drive one. And use HFS+ on your drives. It used to suck, but it has been fixed and works well with System 9.1. Occasionally the HSF+ wrapper will come unbound and you need to fix it with Apple Disk First Aid, though I have not seen that happen since we moved from OS 9 to OS 9.1 so I think they have fixed that issue. Keep the sector size at default unless you really know what you are doing. On backup day one, I slurp the entire contents of drive one to the first set of respective partitions on drive two. On backup day two, I slurp the entire contents of drive one to the second set of respective partitions on drive two. On backup day three, I blow out the first set of backup partitions and slurp the entire contents of drive one over to drive two. And so on. That way I always have two backup copies of the live files -- one completely current and the other several days old. We do not use RAID because there really is no good software RAID systems and a hardware RAID solution is simply too expensive. Plus, with RAID if you make a mistake on a live file, that error is immediately copied over to the backup. Kinda defeats the purpose of having a backup. The timing of the backups is subjective and depends on the overall FTP traffic that has occured since the last backup. For instance, I just backud up the entire system 4 Feb. Prior to that, I backed up 22 Jan since there was very little overall system changes between the two. Things have picked up some and I will do another backup probably this Saturday. During very heavy change seasons like at Christmas, I will back up every day or two. By the way, the entire process takes about four hours and I do everything manually to ensure that it is done absolutely properly. I do not trust automated backup systems at all. One glitch and the whole thing hoses. Many folks are comfortable with writing Apple Scripts to automate the process, though. It is up to you and your comfort level. Recovery is as follows: If the primary drive hoses, I boot off the latest backup drive and the machine is up and running in about three minutes. If the machine completely hoses, I remove the drives and place them in a backup machine that is completely ready to go except for the drives. Back up and running in about 15 minutes. We used to use completely external drives which could be simply unplugged if a machine exploded so you did not have to physically install the drives in the new machine, but again we have had problems with external cases overheating. Also, we have never actually lost a motherboard on a live server. Granted, there is not complete redundency in the event of failure, but doing it the way I do allows me to keep individual hosting account costs to a minimum. If I had full hot swap redundency as well as hardware RAID in place, a basic account would cost about $50 per month and a commerce account would be between $150 and $300 per month plus dramatically increased set up fees. Remember, ultimately the responsibility to back up your files (including database which is why we made it so easy to save as a tab delimited file in MGI 2) is yours. I can generally recover 100 percent of any loss quickly, but there will be times when certain things get lost. Fortunately in almost six years, we have yet to lose any data on a web server apart from one ignorant mistake which I really don't want to talk about. Keep in mind something else about databases in MGI 2 as well. When something gets written to a database, you can also write it to a text file. Under the most dire of circumstances, the integrity of a text file is better than that of a database file.
The machines themselves are all the same:
- Macintosh 7500 or 7600 Sonnet 400 MHz G3 card Asante 10/100 NIC RAM as needed System 9.1 updated current with Software Update WebStar 4.4 MGI 2.whatever (OK, so we need to build a new installer.) Just update it via MGI software update for now. :)
- TB2 5.2.4
- Welcome 3.2
- Remember the old 1967 Dodge Dart with the slant 6 engine that you could drive without oil or water and they would not blow up? That config on that machine is pretty much the same.
- Each machine is plugged into a PowerKeyPro which is plugged into an APC 500 VA UPS unit. Backup power is supplied via a 25KW generator and transfer switch. If the power goes out, I have about 30 minutes to turn on the generator. We are going to get an auto transfer switch when we move the NOC becuse I hate rain and cold which are generally the conditions if the power ever does go out. Whoops. I forgot one of the most important things in speeding up a server and a major security issue to boot."
Steve Crisp's comments are not copyrighted by me, despite the claim in the footer below. ;)
|
Copyright 2002-2005, InterNetyx, All rights reserved
Hits Since 8/00
|
|
|
|
|
|
|