LtG wrote:
If old core is still using the thread resources (kernel stack), then you _may not_ put it in any "free to use" list, including "ready to run", because it's not, its resources are still in use.
One option that you mentioned is to switch to a per-core kernel stack that is used when there's no active userland thread. The alternative is to keep the old kernel stack locked and thus the thread itself locked.
Yes, that is the the problem. However, I'd like to not use it's resources rather than change putting it onto a list. If I was to move putting it onto a queue then, for example, the current CPU wouldn't find it on it's ready to run queue and might select a less important task to run. I'd like to avoid this in an elegant (read: faster, less special cases and branch mispredictions) manner, and not add special checks for the case where the current ask would be ready.
Your first option (my third) is the one I am using currently, but I'd like to know if I'm missing some obvious advantages and disadvantages? And if I'm using per-core stacks anyway is there a good reason to not switch to them entirely? I think this would make Kernel-bound threads difficult/impossible?
Your alternative (my first) means a coupling between CPUs that I think should be unnecessary. I like to try and make all CPUs as separate as possible, to improve scalability. Obviously this problem would only ever effect 2 CPUs though so do you think it would have little impact on scalability?
LtG wrote:
Agalloch wrote:
Which means that between placing the Thread on the list and moving to a new Thread - including all the code to find the next task to run - I will be using the Kernel stack for the thread I placed on the list. Now (multi-CPU), I put the current thread on some kind of list - possibly another CPU's ready to run list - then I search my own ready-to-run list. At this point, I am using the First threads Kernel stack while another CPU might remove the same thread from its own ready to run queue and also begin using the same Stack.
Why are you moving "running thread" from core A to B, just so A can then schedule some other thread? Maybe it's just because of your example, but it would be probably be good to ensure that you never do something like that.. Instead core B should schedule that "some other thread"..
I don't know, it's not relevant to the quesiton. Assume that CPU A is running faster for whatever reason (faster chip, thermal throttling, power states chosen based on temperature etc) and has been preempted by a higher priority task that wants to run on the faster CPU, while CPU B has been considered a reasonable place to continue running the lower priority task. Arbitarily restricting this would reduce performance unncecesarily - the way in which I schedule wouldn't automatically check for this so it'd be an unnecessary extra branch or more. I will think about how easy it is to add though.
LtG wrote:
Agalloch wrote:
2. Guarantee that all code that can be used following a decision to perform a task switch avoids using the Stack. This would mean leaving Interrupts disabled from the moment I decide to yield the current task, through all code to put it on a ready-to-run, sleep or I/O queue, choosing a new Task and saving possibly large amounts of context (FPU/SSE etc).
This is probably going to be a nightmare to maintain, an extra 4KiB (or less?) stack per core is a lot better.
This is my least favourite option also, I just wanted to be thorough in case I missed something.
Thanks!