Multi-threaded

[Sionofdarkness]

Wow brackett that was a lot to take in, how do you know so much?

[amac]

Originally Posted by brackett

I'm not sure what you mean by confusing multi-threading at OS vs process level. Threads are always a part of a process, and whether that process is owned by your application or the OS is immaterial to the discussion, I think.

I don't think it is confusing the issue at all. Firstly processes are always 'owned' by the OS. They are as high level as an application can get. Threads are a part of processes, but multi-threading often refers to what might better be described as multi-processing. Context switching doesn't take place on user-level threads (what we've been calling 'threads' so far).

The point is that kernel level threads (processes) have to perform context switches and lock on shared data, but user-level threads don't. They exist for this reason - efficiency!. So you don't get a context switching penalty, and that's a pretty major point. I'm not saying talking about multi-processing is irrelevant; just that you need to separate it from the issue of user level threads.

Writing a multi-threaded program where user input is required (as well as processing) is much, much simpler than writing a single-threaded one. Java abstracts from any complexity, and doesn't resort to mindless polling etc. How would you go about breaking "long running process in manageable chunks and poll the input in between chunks" whilst still getting all user input? It would be insane. It's not multi-threading thats inefficient - polling is.

To poll for user input wastes ridiculous amounts of cycles; using multiple threads allows the program to fall back on the OS's interrupts meaning that the thread doesn't do anything when it doesn't have to. So it's not less efficient, and I think it's important to remember that. If you poll, as you suggest, you're essentially implementing your own basic threading system - you perform some action, go off and do another, then come back to the first one. Why do that when you've got an optimised version already available? To sum up... never, never, never poll for user input (and if you do, please don't mention efficiency in the same sentence).

[amac]

Originally Posted by Sionofdarkness

Wow brackett that was a lot to take in, how do you know so much?

He doesn't. He really doesn't.

[brackett]

Originally Posted by amac

I don't think it is confusing the issue at all. Firstly processes are always 'owned' by the OS. They are as high level as an application can get. Threads are a part of processes, but multi-threading often refers to what might better be described as multi-processing.

According to Wikipedia, multi-processing refers to using multiple CPUs to execute simultaneously - which is where this discussion started. Without multiple CPUs (or cores), you're really just switching tasks on a single CPU.

The discussion is, I think, when and whether multi-threading is advantageous. My argument is that it's actually *not* that much of an advantage now (because you're not "multi-processing", since 95+% of desktops are single core) but that it will become increasingly important as dual and quad cores become common place. Also, there's both a (slight) performance penalty, and a increased maintenance cost with multithreaded code. All of that needs to be considered.

Originally Posted by amac

Context switching doesn't take place on user-level threads (what we've been calling 'threads' so far).

Hmmm...how does the processor save and load the stack (among other things) to execute a new thread then? I assume you're "user-level threads" are what I term fibers? Cooperatively scheduled threads, with scheduling handled in userspace (as opposed to preemptively scheduled threads, with scheduling handled in the kernel)? While it's true that a context switch between processes is heavier than a kernel thread, and a switch between kernel threads is heavier than a user thread, to say there's no context switch penalty is off the mark.

Originally Posted by amac

The point is that kernel level threads (processes) have to perform context switches and lock on shared data, but user-level threads don't. They exist for this reason - efficiency!.

Kernel level threads are not processes - they are the normal multi-threading model in the languages and systems I am familiar with. AFIAK, Windows, Linux, .NET, Java, and even Solaris all use kernel level threads. (To be fair, earlier implementations of Solaris, and the JVM used fibers, but their current implementations do not. Also, Windows NT has a fiber API to compete with earlier Solaris versions, but it's use is strongly discouraged. I *think* that Ruby uses fibers, or some sort of hybrid model).

If there is no need for locking on shared data, somebody should tell the Java folks that the synchronized keyword is no longer needed.

Originally Posted by amac

Writing a multi-threaded program where user input is required (as well as processing) is much, much simpler than writing a single-threaded one. Java abstracts from any complexity, and doesn't resort to mindless polling etc. How would you go about breaking "long running process in manageable chunks and poll the input in between chunks" whilst still getting all user input? It would be insane. It's not multi-threading thats inefficient - polling is.

In the example given, you're actually right. Since you're talking about getting input from a keyboard, you're able to set it up with an IRQ handler. I ran with the example given, and botched it - mea culpa. My point, however, that multi-threading is *not* (generally) faster on a single CPU system remains.

Originally Posted by amac

He doesn't. He really doesn't.

While I have no problem with you disagreeing with me, and even pointing out that I'm a blathering idiot - I'd prefer you do it within the context of the discussion.

I didn't make it a point to show that you confused "kernel-level threads" with processes, and stated that "processes" need to lock on shared data (they don't - processes communicate via IPC, and have no shared data) or that you imply that you don't need to lock on shared data within Java (a dangerous point of view, BTW). Instead, I simply replied to your arguments. I'd appreciate it if you kept the same civility.

[amac]

Originally Posted by brackett

According to Wikipedia, multi-processing refers to using multiple CPUs to execute simultaneously - which is where this discussion started. Without multiple CPUs (or cores), you're really just switching tasks on a single CPU.

The discussion is, I think, when and whether multi-threading is advantageous. My argument is that it's actually *not* that much of an advantage now (because you're not "multi-processing", since 95+% of desktops are single core) but that it will become increasingly important as dual and quad cores become common place. Also, there's both a (slight) performance penalty, and a increased maintenance cost with multithreaded code. All of that needs to be considered..

Well yes. I said 'might better be described as multi-processing'. Not 'this is multi-processing'.

When you say that multi-threading is not faster on a single CPU. Not faster than what? I couldn't give you an example of a program that uses multiple threads that could be better written in a single thread; because as I said before, you resort to polling or you start rejecting input.

Imagine writing a web-browser in a single thread. Aside from the size of the code-base, how would you handle connections to multiple web-pages without polling? My point is, that in single core processors, threads allow programs to abstract away the difficulty of scheduling multiple jobs. You couldn't write a faster single threaded version. So, no, multi-threading is not slower on single core CPU's (unless you mis-use them).

Originally Posted by brackett

If there is no need for locking on shared data, somebody should tell the Java folks that the synchronized keyword is no longer needed.

Okay, the way I said that wasn't particuarly clear. They are required, but thread context switching is exceptionally minimal compared to process context switching. Despite the cost there's no better alternative, so its not an issue.

Originally Posted by brackett

My point, however, that multi-threading is *not* (generally) faster on a single CPU system remains.

It's slower where it isn't needed. But in programs that require multiple connections (or similar) it isn't possible to do it more efficient any other way.

[WingedPanther]

Multi-Threading on a multi-core processor can give you tremendous speed advantanges, but multi-threading in any environment can allow you to perform certain tasks that just aren't as easy to make work right any other way.

[brackett]

Originally Posted by amac

When you say that multi-threading is not faster on a single CPU. Not faster than what?

If you're writing a network server, you have a couple of ways of handling multiple requests. You can implement 1 thread per connection with blocking sockets (or, in *NIX, you'd normally do a fork and have 1 process per connection), or you can have an array of non blocking sockets and use a single thread to select() from them. If you use the select() method, you can then either handle the client inline, start a new worker thread to handle the client, or use a threadpool to service the client.

Traditionally, select() (combined with inline handling, or a threadpool - mainly depending on how long you'd be talking to the client) would be the most performant and scalable (actually, I/O completion ports would be better under Win32, but only because select() is poorly implemented). At a couple of hundred connections, having a single thread per connection will result in your app spending most of its time in thread management (starting, stopping, or switching).

But, back to my original point. multicore CPUs start to tip the balance back towards multithreading, as you can now utilize both cores (the Apache model of using multiple processes, each handling a set of sockets with a select() call, would also work - but is probably more complicated to design).

Originally Posted by amac

I couldn't give you an example of a program that uses multiple threads that could be better written in a single thread; because as I said before, you resort to polling or you start rejecting input.

And what do you think your second thread has to do (excepting an IRQ handler, as we discussed before)? Having a second thread waiting on something doesn't remove polling or blocking, it just switches it to another thread. As a bonus, the CPU now has to wake up that other thread every once in a while to see if it's ready to do something.

The basic fact is that a single core CPU can only run a single thread at a time (excepting HyperThreading, which comes with its own set of rules). There's no way you can get around that. So, in effect, you're just splitting out half your logic, forcing the CPU to switch between the 2, and have to worry about shared data, locking, and synchronization (whether or not that's a big deal depends on your design and app, of course).

Originally Posted by amac

Imagine writing a web-browser in a single thread. Aside from the size of the code-base, how would you handle connections to multiple web-pages without polling?

Non blocking sockets. And I don't see how it would increase the size of the code base at all - in fact, since you don't have to deal with locking and synchronization, it'd probably be smaller.

I'm not familiar with the internals of any web browser, but I suppose you could dig up any browser written for DOS or Win 3.1 and see a single threaded browser in action.

Note that I *wouldn't* necessarily design it that way. I think that a web browser design would probably lend itself nicely to a mutlithreaded design; I actually use multiple threads quite a bit - but I am aware of the limitations and alternatives.

Originally Posted by amac

My point is, that in single core processors, threads allow programs to abstract away the difficulty of scheduling multiple jobs.

And abstraction is a fine goal - but, as in most things abstracted, you lose out on performance. In multithreaded, you also (usually) lose out on maintenance.

Originally Posted by amac

It's slower where it isn't needed. But in programs that require multiple connections (or similar) it isn't possible to do it more efficient any other way.

It's never *needed*. Witness all of the single threaded apps produced for DOS, Win 3.1, and (most) embedded systems (all of which do not support multithreading).

It's a design choice. And, like all design choices, it's gotta be made in context. My original point was that multicore CPUs change the balance, and multithreading will likely become more common and the correct choice more often.

Unfortunately, a lot of programmers do not understand the difficulties in multithreading, locking, and synchronization. Mainly because they run on a single core CPU which (since it really only runs 1 thread at a time) masks a lot of the bugs in their code. Move that to a multicore CPU, and all of those data corruption issues, deadlocking, etc. will happen a lot more often.