setting up a subnet work??

[T Molnar]

  I wish to set up a network in my house with internet access via a 
cable modem.

I will be hooking up my tibook, an old supermac s900 and perhaps 2 
recent windows machines (people living up stairs).
The cable co. will give me 2 static ip addresses.
In the past i had 2 computers running through a hub each with their own 
ip address, that was easy to set up.

But for this, do i need some sort of addressing router (?) to assign a 
temporary address.
I am not sure how I go about addressing additional machines so they can 
all have access.

Any input is much appreciated.

cheers,
tim