[X-Unix] syslogd problems

[William H. Magill]

On 08 Jun, 2004, at 13:17, Kevin Stevens wrote:
> Or I could just pull them from my FreeBSD box.  ;)  Didn't realize they
> were the same.  Does that mean that I should be able to cross-reference
> userland executable issues in general, or just expect the same
> functionality?  FreeBSD 4.x or 5.x?
>
> ... (checking)
> Well, I already see one potential problem.  -u per the BSD man page
> enables unique priority logging, while per the OS X man page it sets 
> UDP
> mode!  No wonder I was having trouble getting it to work!
> grr...

It depends.
Apple does not simply implement ALL of FreeBSD. Nor is it "current." I 
believe Darwin is now "based" on 4.5, but I'd have to go through the 
Darwin website and read up.

> Maybe.  But by "unresponsive" I mean that the mouse will move around 
> the
> screen, but won't activate/bring forward different running windows 
> (though
> the window contents, for example 'top', keep refreshing).  That doesn't
> seem very disk-ish.  How would I check, some incantation of vmstat or
> iostat?

Sounds more like a memory subsystem issue.

I don't have enough experience with the memory management scheme used 
in Darwin, and I've never read an "academic" analysis of it, or 
comparison with other schemes, so this is speculation based on 
experience with other virtual memory management schemes and their 
problems. Virtual memory is not as much a "done deal" as it seems to 
be. It means different things to different people, and it is 
implemented differently, as well as with different assumptions by 
virtually every OS.

 From comments I've read and some empirical information, it appears that 
Darwin attempts to keep "maximum stuff for a program in physical 
memory" if at all possible. Possibly with minimal garbage collection.

One problem which memory management schemes exhibit is that they 
perform differently, frequently significantly, when only one program is 
running compared to when multiple programs are contesting for 
resources. "One program" really means -- N-programs, all of which fit 
into physical memory. These characteristics are also influenced by the 
design and construction of the individual programs involved. 
Frequently, the "expectations" that the user has about how a program 
works is quite different from those of the original programmer.

That is to say, if there is only one program running, and it has a 
large data space (like syslogd would after a time running) it could 
completely fill all of physical memory and all of swap, even though 
much or most of the space is occupied by data which will not be 
referenced again... typical of a logging situation. This means that for 
a new program to launch, the memory subsystem first has to do 
significant garbage collection and cleanup.

One "user expectation" about syslogd is that it is a very limited 
program in its resource demands. However, since it is written to be a 
"dynamic" program, which will not suddenly fail because the volume of 
data for its buffers exceeds what the programmer decided was "a 
reasonable amount," that is undoubtedly an "unreasonable" assumption.

I suspect that the only thing which will tell you what is happening is 
something like the old unix Profiler application, which requirers that 
the hooks be compiled in.  If Darwin implemented System Accounting, it 
might be possible to find out fairly painlessly what any given 
application was actually doing and how it was interacting with the 
operating system. The problem is that such things generate sufficient 
overhead that they are eliminated from virtually all modern operating 
system implementations to enhance performance. (Not to mention the fact 
that their usage is more than slightly arcane.) Consequently one has to 
find and install them and hope that they actually do integrate 
correctly.

In the end, suspect you will discover that you are not seeing a bug, 
but rather "unexpected, but correct" behavior.

One clue would be ... if you play with the system in its sluggish 
state, does its performance improve over time? ... where time is a 
parameter defined in the same sense that the timers in SMTP are 
defined. (Which are NOT as short as most people assume them to be.)

T.T.F.N.
William H. Magill
# Beige G3 - Rev A motherboard - 768 Meg
# Flat-panel iMac (2.1) 800MHz - Super Drive - 768 Meg
# PWS433a [Alpha 21164 Rev 7.2 (EV56)- 64 Meg]- Tru64 5.1a
# XP1000  [Alpha EV6]
magill at mcgillsociety.org
magill at acm.org
magill at mac.com