Server - tasking and long lived connections

Server - tasking and long lived connections

[Eric Merritt]

Hello All,

I am in the process of designing a server that will
need to accept long lived (up to several hours) client
connections. The processes themselves will be fairly
I/O bound (database accesses), and at most the number
of processors will be in the range of 2 - 4. The
obvious choice (in my mind) for this is to use a
single task per connection. In this senario, context
switching will not be a major issues due to the length
of the connections. There will be little or no
performance gains by using tasking, but it handles the
long connection times well. 

I do, however, have a few reservations about
scalablity due to OS limitations on the number of
threads per machine and the number of open file
descriptors. Although at first the number of
connections will not be high, it is possible that it
could rapidly need to scale up to a fairly high
number, several hundred or more. This may mean that I
will just have to distribute the app across several
boxs and do some type of mirroring on the databases,
but I want to put that off as long as possible. 

I have quit a bit of experience in code this type of
application but usually with short lived connections
or a low number of total connections. 

Anyway, I am just looking for some one to either point
me in a different direction or confirm my current
approach (one thread per connection).

Thank you for your input.


__________________________________________________
Do You Yahoo!?
Check out Yahoo! Shopping and Yahoo! Auctions for all of
your unique holiday gifts! Buy at http://shopping.yahoo.com
or bid at http://auctions.yahoo.com

Re: Server - tasking and long lived connections

[James Rogers]

Hi Eric.

Which operating system are you targeting for this application?

WinNT easily handles about 2000 of tasks. I do not know the
limits on open file descriptors.

The tasking approach is somewhat more efficient than the traditional
Unix approach of creating a separate process to handle each 
connection. The tasking approach will require the same number
of file descriptors and data base connections. Context switching will
be faster with tasks than with processes.

Which other design approaches would you consider if your operating
system resources are too limited? Would you be able to expand the
system by creating a set of application servers all accessing
a central database through a network? Would you need a system that
performed automatic load balancing across a network?

One advantage of the tasking approach is that you will be able to
create the tasking code as a task type, then create as many 
instances of that task type as you need. From the Ada point of
view this solution is highly scalable.

Jim Rogers

Eric Merritt wrote:
> 
> Hello All,
> 
> I am in the process of designing a server that will
> need to accept long lived (up to several hours) client
> connections. The processes themselves will be fairly
> I/O bound (database accesses), and at most the number
> of processors will be in the range of 2 - 4. The
> obvious choice (in my mind) for this is to use a
> single task per connection. In this senario, context
> switching will not be a major issues due to the length
> of the connections. There will be little or no
> performance gains by using tasking, but it handles the
> long connection times well.
> 
> I do, however, have a few reservations about
> scalablity due to OS limitations on the number of
> threads per machine and the number of open file
> descriptors. Although at first the number of
> connections will not be high, it is possible that it
> could rapidly need to scale up to a fairly high
> number, several hundred or more. This may mean that I
> will just have to distribute the app across several
> boxs and do some type of mirroring on the databases,
> but I want to put that off as long as possible.
> 
> I have quit a bit of experience in code this type of
> application but usually with short lived connections
> or a low number of total connections.
> 
> Anyway, I am just looking for some one to either point
> me in a different direction or confirm my current
> approach (one thread per connection).
> 
> Thank you for your input.
> 
> __________________________________________________
> Do You Yahoo!?
> Check out Yahoo! Shopping and Yahoo! Auctions for all of
> your unique holiday gifts! Buy at http://shopping.yahoo.com
> or bid at http://auctions.yahoo.com

Re: Server - tasking and long lived connections

[Eric Merritt]

Thanks Jim I guess I should have asked you first.

> Which operating system are you targeting for this
> application?
 
 To begin with the *nix machines, mainly linux and
FreeBsd, but it could easily make its way to windows,
solaris, hp-ux, etc depending on whether its
opensourced in the end. That is one of the reasons my
arguments for Ada won out.

> WinNT easily handles about 2000 of tasks. I do not
> know the
> limits on open file descriptors.

 Linux threading is a bit less efficient from what I
understand. I think it by default can handle about 512
tasks, depending on configuration. This can be changed
with a custom kernel but even then it starts to get
inefficient after a certain point.

 
> The tasking approach is somewhat more efficient than
> the traditional Unix approach of creating a separate
process to
> handle each 
> connection. The tasking approach will require the
> same number
> of file descriptors and data base connections.
> Context switching will
> be faster with tasks than with processes.

Using processes never really crossed my mind, I tend
not to really consider them becuase they have no real
advantage over threads and alot of disadvantages.

> Which other design approaches would you consider if
> your operating
> system resources are too limited? 
> Would you be able
> to expand the
> system by creating a set of application servers all
> accessing
> a central database through a network? 

That is the main option I have in mind at the moment
or perhaps realtime datasync between databases on each
boxe, I really hope it doesn't get to this point.

> Would you need
> a system that
> performed automatic load balancing across a network?

If I had to move to seperate servers this would really
need to be considered carefully. 
 
> One advantage of the tasking approach is that you
> will be able to
> create the tasking code as a task type, then create
> as many 
> instances of that task type as you need. From the
> Ada point of
> view this solution is highly scalable.
> 
 I like the tasking approach, it is conceptually very
simple in this type of application, and from a
development perspective is very maintainable.



__________________________________________________
Do You Yahoo!?
Check out Yahoo! Shopping and Yahoo! Auctions for all of
your unique holiday gifts! Buy at http://shopping.yahoo.com
or bid at http://auctions.yahoo.com

Re: Server - tasking and long lived connections

[Florian Weimer]

James Rogers <jimmaureenrogers@worldnet.att.net> writes:

> The tasking approach is somewhat more efficient than the traditional
> Unix approach of creating a separate process to handle each 
> connection.

For long-living connections, this might not be the case.  It depends
on several factors.  For example, if there isn't much communication
required between the theads of control handling separate connections,
and each thread of control performs many memory allocations and
deallocations, the separate process implementation is probably faster.

> Context switching will be faster with tasks than with processes.

This depends on the OS and Ada implementation.

Re: Server - tasking and long lived connections

[Florian Weimer]

Eric Merritt <cyberlync@yahoo.com> writes:

>  Linux threading is a bit less efficient from what I
> understand. I think it by default can handle about 512
> tasks, depending on configuration. This can be changed
> with a custom kernel but even then it starts to get
> inefficient after a certain point.

The 512 tasks limit is probably related to some address space limit,
since each task needs address space for its stack (2 MB by default
with GNU libc pthreads, IIRC).

There are several threading implementations for GNU/Linux (GNU libc,
FSU Threads, GNU Pth, GNU Pth with IBM extensions).  Some of the scale
better than others.

How many parallel connections do you want to handle?  Is data
transferred continuously?  Does your database backend support so many
client connections?

> Using processes never really crossed my mind, I tend
> not to really consider them becuase they have no real
> advantage over threads and alot of disadvantages.

There are advantages.  For example, they are more portable ac

Re: Server - tasking and long lived connections

[Larry Kilgallen]

In article <mailman.1008382022.25904.comp.lang.ada@ada.eu.org>, Eric Merritt <cyberlync@yahoo.com> writes:

> I do, however, have a few reservations about
> scalablity due to OS limitations on the number of
> threads per machine and the number of open file
> descriptors.

This must enter into your choice of operating system, but I don't
think of the term "file descriptors" as being an Ada phrase.  If
you are limiting your choice to operating systems that use such
a construct, consider whether that is an artificial limit on the
available implementations.

The operating system with which I am most familiar had to increase
the number of simultaneous IO channels available, in recent years.
So not only the operating system but also the version may be important.

In that case, there are also multiple network implementations
available for each of two protocol sets.  These also have different
limitations.

Re: Server - tasking and long lived connections

[Steve Doiel]

I certainly wouldn't rule out a tasking arrangement using one task per N
file descriptors,  where the number of file descriptors handled for each
task is target specific.  I know that the "select" function on some targets
have a limit on the number of file descriptors, while other targets have a
limit on the number of tasks that may be used.

Our applications run on Winows NT 4.0.  We use more than one task for each
connection (input, output and connection maintainence).  We typically have a
few dozen connections.  This implementation proves to be very simple and
easy to maintain.  But you're talking about a lot more connections.

SteveD

"Eric Merritt" <cyberlync@yahoo.com> wrote in message
news:mailman.1008382022.25904.comp.lang.ada@ada.eu.org...
> Hello All,
>
> I am in the process of designing a server that will
> need to accept long lived (up to several hours) client
> connections. The processes themselves will be fairly
> I/O bound (database accesses), and at most the number
> of processors will be in the range of 2 - 4. The
> obvious choice (in my mind) for this is to use a
> single task per connection. In this senario, context
> switching will not be a major issues due to the length
> of the connections. There will be little or no
> performance gains by using tasking, but it handles the
> long connection times well.
>
> I do, however, have a few reservations about
> scalablity due to OS limitations on the number of
> threads per machine and the number of open file
> descriptors. Although at first the number of
> connections will not be high, it is possible that it
> could rapidly need to scale up to a fairly high
> number, several hundred or more. This may mean that I
> will just have to distribute the app across several
> boxs and do some type of mirroring on the databases,
> but I want to put that off as long as possible.
>
> I have quit a bit of experience in code this type of
> application but usually with short lived connections
> or a low number of total connections.
>
> Anyway, I am just looking for some one to either point
> me in a different direction or confirm my current
> approach (one thread per connection).
>
> Thank you for your input.
>
>
> __________________________________________________
> Do You Yahoo!?
> Check out Yahoo! Shopping and Yahoo! Auctions for all of
> your unique holiday gifts! Buy at http://shopping.yahoo.com
> or bid at http://auctions.yahoo.com

Re: Server - tasking and long lived connections

[Eric Merritt]

> How many parallel connections do you want to handle?
>  Is data
> transferred continuously?  Does your database
> backend support so many
> client connections?

Data transfer : There will probably be small data
transfers on the order of tens or hundreds of bytes
more or less continuously. 

Parallel Connections : thirty or forty probably at the
lowest end, as many as passable at the highest end.

I realize that range is vague, I am pretty sure that
if the number of connections is very high then I will
inevitably have to go to a multiple server approach.
My goal in this case is to delay that as long as
possible. Efficiency is a goal in and of itself as
well.

Database Client Connections : I plan to implement
connection pooling for the databases so I should never
really have more then a handful of open connections
the database depending on how many concurrent users
there.



__________________________________________________
Do You Yahoo!?
Check out Yahoo! Shopping and Yahoo! Auctions for all of
your unique holiday gifts! Buy at http://shopping.yahoo.com
or bid at http://auctions.yahoo.com

RE: Server - tasking and long lived connections

[Steven Deller]

In the early 90's I consulted on a system that created more that 10,000
tasks in Ada.  The problem was that each task used 100KB of stack and
the 1GB of memory just for stack meant they could not load the
application.

That was solved by making a few more tasks, each for the "computational"
aspects of the problem.  The "computation" tasks had 100KB or more for
their stack.  The 10,000 "interface" tasks wre then able to operate with
only 5KB of stack -- virtually no "depth" to calls.  The "interface"
tasks simply collected data (asynchronously) then rendezvous'ed with one
or more computational tasks to actually process the data.

Besides allowing the application to fit in memory, the transformation
also improved throughput by more than 2x because only the computational
tasks used floating point so task switching often did not have to save
and reload floating point registers.

Such a transformation might be feasible for your application so if you
can control the stack size of tasks with your chosen compiler and OS,
then likely the only limit would be some artificial OS limit on the
number of threads per process.

Regards,
Steven Deller 

> -----Original Message-----
> From: comp.lang.ada-admin@ada.eu.org 
> [mailto:comp.lang.ada-admin@ada.eu.org] On Behalf Of Eric Merritt
> Sent: Saturday, December 15, 2001 12:56 PM
> To: comp.lang.ada@ada.eu.org
> Subject: Re: Server - tasking and long lived connections
> 
> 
> > How many parallel connections do you want to handle?
> >  Is data
> > transferred continuously?  Does your database
> > backend support so many
> > client connections?
> 
> Data transfer : There will probably be small data
> transfers on the order of tens or hundreds of bytes
> more or less continuously. 
> 
> Parallel Connections : thirty or forty probably at the
> lowest end, as many as passable at the highest end.
> 
> I realize that range is vague, I am pretty sure that
> if the number of connections is very high then I will 
> inevitably have to go to a multiple server approach. My goal 
> in this case is to delay that as long as possible. Efficiency 
> is a goal in and of itself as well.

RE: Server - tasking and long lived connections

[Eric Merritt]

This sounds like a pretty good idea, I will have to
think about it in reference to this application.

--- Steven Deller <deller@smsail.com> wrote:
> In the early 90's I consulted on a system that
> created more that 10,000
> tasks in Ada.  The problem was that each task used
> 100KB of stack and
> the 1GB of memory just for stack meant they could
> not load the
> application.
> 
> That was solved by making a few more tasks, each for
> the "computational"
> aspects of the problem.  The "computation" tasks had
> 100KB or more for
> their stack.  The 10,000 "interface" tasks wre then
> able to operate with
> only 5KB of stack -- virtually no "depth" to calls. 
> The "interface"
> tasks simply collected data (asynchronously) then
> rendezvous'ed with one
> or more computational tasks to actually process the
> data.
> 
> Besides allowing the application to fit in memory,
> the transformation
> also improved throughput by more than 2x because
> only the computational
> tasks used floating point so task switching often
> did not have to save
> and reload floating point registers.
> 
> Such a transformation might be feasible for your
> application so if you
> can control the stack size of tasks with your chosen
> compiler and OS,
> then likely the only limit would be some artificial
> OS limit on the
> number of threads per process.
> 
> Regards,
> Steven Deller 
> 
> > -----Original Message-----
> > From: comp.lang.ada-admin@ada.eu.org 
> > [mailto:comp.lang.ada-admin@ada.eu.org] On Behalf
> Of Eric Merritt
> > Sent: Saturday, December 15, 2001 12:56 PM
> > To: comp.lang.ada@ada.eu.org
> > Subject: Re: Server - tasking and long lived
> connections
> > 
> > 
> > > How many parallel connections do you want to
> handle?
> > >  Is data
> > > transferred continuously?  Does your database
> > > backend support so many
> > > client connections?
> > 
> > Data transfer : There will probably be small data
> > transfers on the order of tens or hundreds of
> bytes
> > more or less continuously. 
> > 
> > Parallel Connections : thirty or forty probably at
> the
> > lowest end, as many as passable at the highest
> end.
> > 
> > I realize that range is vague, I am pretty sure
> that
> > if the number of connections is very high then I
> will 
> > inevitably have to go to a multiple server
> approach. My goal 
> > in this case is to delay that as long as possible.
> Efficiency 
> > is a goal in and of itself as well.
> 
> _______________________________________________
> comp.lang.ada mailing list
> comp.lang.ada@ada.eu.org
> http://ada.eu.org/mailman/listinfo/comp.lang.ada


__________________________________________________
Do You Yahoo!?
Check out Yahoo! Shopping and Yahoo! Auctions for all of
your unique holiday gifts! Buy at http://shopping.yahoo.com
or bid at http://auctions.yahoo.com

Re: Server - tasking and long lived connections

[Thierry Lelegard]

We have applications that we successfully tested with up to 6000 tasks.
That was on OpenVMS with the DEC Ada compiler. We now use GNAT on
VMS/UNIX/WNT. We haven't yet made such load testing but several hundredths
of tasks run fine.

On most operating systems, though, this requires some tuning (memory
management, I/O, etc).

I think that the trap in this kind of applications (servers with one
or more tasks per client connection) is the fate of the tasks when
the client disconnects. If you let the tasks die and recreate new
tasks for new client connections, you will most certainly face some
slow memory leak. We have seen that, although the stack itself is
deallocated when a task dies, there aer a few bytes which remain
allocated (some kind of TCB I suppose).

So, the trick is to never let the service tasks die. When a client
disconnects, just queue the service task for future usage by a new
client connection. When a new connection comes in, reuse an idle
service task. Create a new one only when there is no available
idle task.

Of course, if you connection are long lived (several hours) and your
application is relatively short lived (a few weeks), this may not
be an issue. But if you want your application to potentially run
forever, you should take care.

In the (long) past, I have seen a Minitel server (French ancestor
of the WWW) crashing every 3 months, due to virtual memory exhaustion,
just because the Ada tasks died when the clients disconnected.

-Thierry
____________________________________________________________________________

Thierry Lelegard, "The Jazzing Troll", Email: thierry.lelegard@canal-plus.fr
CANAL+ Technologies, 34 place Raoul Dautry, 75906 Paris Cedex 15, France
Tel: +33 1 71 71 54 30   Fax: +33 1 71 71 52 08   Mobile: +33 6 03 00 65 75
____________________________________________________________________________

Re: Server - tasking and long lived connections

[Jean-Pierre Rosen]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 879 bytes --]


"Thierry Lelegard" <thierry.lelegard@canal-plus.fr> a �crit dans le message news: 3C1DB7B7.DF767F9@canal-plus.fr...
> I think that the trap in this kind of applications (servers with one
> or more tasks per client connection) is the fate of the tasks when
> the client disconnects. If you let the tasks die and recreate new
> tasks for new client connections, you will most certainly face some
> slow memory leak. We have seen that, although the stack itself is
> deallocated when a task dies, there aer a few bytes which remain
> allocated (some kind of TCB I suppose).
>
This was due to the (in)famous Rosen's pathology ;-)
It has been (fortunately) exterminated in in Ada 95, so the problem should not appear any more.

--
---------------------------------------------------------
           J-P. Rosen (rosen@adalog.fr)
Visit Adalog's web site at http://www.adalog.fr

Re: Server - tasking and long lived connections

[Thierry Lelegard]

Jean-Pierre Rosen wrote :
> 
> "Thierry Lelegard" <thierry.lelegard@canal-plus.fr> a �crit dans le message news: 3C1DB7B7.DF767F9@canal-plus.fr...
> > I think that the trap in this kind of applications (servers with one
> > or more tasks per client connection) is the fate of the tasks when
> > the client disconnects. If you let the tasks die and recreate new
> > tasks for new client connections, you will most certainly face some
> > slow memory leak. We have seen that, although the stack itself is
> > deallocated when a task dies, there aer a few bytes which remain
> > allocated (some kind of TCB I suppose).
> >
> This was due to the (in)famous Rosen's pathology ;-)
> It has been (fortunately) exterminated in in Ada 95, so the problem should not appear any more.

Quite interesting. Could you elaborate on this? What kind of change was
introduced in Ada95 to eliminate this?

-Thierry
____________________________________________________________________________

Thierry Lelegard, "The Jazzing Troll", Email: thierry.lelegard@canal-plus.fr
CANAL+ Technologies, 34 place Raoul Dautry, 75906 Paris Cedex 15, France
Tel: +33 1 71 71 54 30   Fax: +33 1 71 71 52 08   Mobile: +33 6 03 00 65 75
____________________________________________________________________________

Re: Server - tasking and long lived connections

[Larry Kilgallen]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 966 bytes --]

In article <9vkecb$a6v$1@s1.read.news.oleane.net>, "Jean-Pierre Rosen" <rosen@adalog.fr> writes:
> 
> "Thierry Lelegard" <thierry.lelegard@canal-plus.fr> a �crit dans le message news: 3C1DB7B7.DF767F9@canal-plus.fr...
>> I think that the trap in this kind of applications (servers with one
>> or more tasks per client connection) is the fate of the tasks when
>> the client disconnects. If you let the tasks die and recreate new
>> tasks for new client connections, you will most certainly face some
>> slow memory leak. We have seen that, although the stack itself is
>> deallocated when a task dies, there aer a few bytes which remain
>> allocated (some kind of TCB I suppose).
>>
> This was due to the (in)famous Rosen's pathology ;-)
> It has been (fortunately) exterminated in in Ada 95, so the problem should not appear any more.

Does the standard mandate that such a problem not happen,
or simply avoid mandating something that would force it to happen?

Re: Server - tasking and long lived connections

[Pat Rogers]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1535 bytes --]


"Larry Kilgallen" <Kilgallen@SpamCop.net> wrote in message
news:YaH+t3eYcXsB@eisner.encompasserve.org...
> In article <9vkecb$a6v$1@s1.read.news.oleane.net>, "Jean-Pierre Rosen"
<rosen@adalog.fr> writes:
> >
> > "Thierry Lelegard" <thierry.lelegard@canal-plus.fr> a �crit dans le
message news: 3C1DB7B7.DF767F9@canal-plus.fr...
> >> I think that the trap in this kind of applications (servers with one
> >> or more tasks per client connection) is the fate of the tasks when
> >> the client disconnects. If you let the tasks die and recreate new
> >> tasks for new client connections, you will most certainly face some
> >> slow memory leak. We have seen that, although the stack itself is
> >> deallocated when a task dies, there aer a few bytes which remain
> >> allocated (some kind of TCB I suppose).
> >>
> > This was due to the (in)famous Rosen's pathology ;-)
> > It has been (fortunately) exterminated in in Ada 95, so the problem
should not appear any more.
>
> Does the standard mandate that such a problem not happen,
> or simply avoid mandating something that would force it to happen?

It is solved indirectly because functions can no longer return limited types
(eg task types) by-value.  (A nice simple rule that solves a number of
issues.)

---
Patrick Rogers                       Consulting and Training in:
http://www.classwide.com          Real-Time/OO Languages
progers@classwide.com               Hard Deadline Schedulability Analysis
(281)648-3165                                 Software Fault Tolerance

Re: Server - tasking and long lived connections

[Jean-Pierre Rosen]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1931 bytes --]


"Thierry Lelegard" <thierry.lelegard@canal-plus.fr> a �crit dans le message news: 3C1DC5ED.9E9F11F1@canal-plus.fr...
> Jean-Pierre Rosen wrote :
> >
> > "Thierry Lelegard" <thierry.lelegard@canal-plus.fr> a �crit dans le message news: 3C1DB7B7.DF767F9@canal-plus.fr...
> > > I think that the trap in this kind of applications (servers with one
> > > or more tasks per client connection) is the fate of the tasks when
> > > the client disconnects. If you let the tasks die and recreate new
> > > tasks for new client connections, you will most certainly face some
> > > slow memory leak. We have seen that, although the stack itself is
> > > deallocated when a task dies, there aer a few bytes which remain
> > > allocated (some kind of TCB I suppose).
> > >
> > This was due to the (in)famous Rosen's pathology ;-)
> > It has been (fortunately) exterminated in in Ada 95, so the problem should not appear any more.
>
> Quite interesting. Could you elaborate on this? What kind of change was
> introduced in Ada95 to eliminate this?
>
Here is (was) the problem:

task type TT is....

function F return TT is
   Local : tt;
begin
   return Local;
end F;

....

if F'Terminated then..... -- Allways true

The master of Local is function F; but since it is returned by the function, it is still possible to access the task object after
the scope of its master has been left (although the task will allways be terminated); hence the need to keep a minimum information,
and since the master is left, there is no logical place where that information can be reclaimed.

In Ada 95, a task type is a by-reference type (6.2(6)), and a function is not allowed to return a local object of a by-reference
type (6.5(18)). Therefore, the above program will raise Program_Error (6.5(20)).

--
---------------------------------------------------------
           J-P. Rosen (rosen@adalog.fr)
Visit Adalog's web site at http://www.adalog.fr

Re: Server - tasking and long lived connections

[Matthew Heaney]

"Eric Merritt" <cyberlync@yahoo.com> wrote in message
news:mailman.1008382022.25904.comp.lang.ada@ada.eu.org...
> Anyway, I am just looking for some one to either point
> me in a different direction or confirm my current
> approach (one thread per connection).

One thread per connection cannot possibly scale well.  It is the *wrong* way
to implement a server.

Our RTSP video server uses WinNT I/O completion ports, so that a small
number of tasks (threads) can service hundreds (or even thousands) of client
connections.

Re: Server - tasking and long lived connections

[Eric Merritt]

> One thread per connection cannot possibly scale
> well.  It is the *wrong* way
> to implement a server.
> 
> Our RTSP video server uses WinNT I/O completion
> ports, so that a small
> number of tasks (threads) can service hundreds (or
> even thousands) of client
> connections.

Actually after thinking about it I agree with you.
Though the connections are long lived (on the order of
several hours) that doesn't really remove the
usefulness of a thread pool, it just makes it less
valuable. It makes it less valuable because context
switching takes up a much smaller percentage of
overall run time. Even in this case a thread pool is
still a good idea.


__________________________________________________
Do You Yahoo!?
Check out Yahoo! Shopping and Yahoo! Auctions for all of
your unique holiday gifts! Buy at http://shopping.yahoo.com
or bid at http://auctions.yahoo.com