* locking

regulate access to some shared d-s.

Many different kinds of locks:
1. fast vs. slow locks
2. blocking vs. non-blocking locks.
3. exclusive locks (only one at a time)
4. non-exclusive locks (multiple "users" can hold a lock).
5. "lock-less" locks...

A lock has to perform some ATOMIC action, guaranteed that only one will
succeed and all others will fail.  Can't be done in s/w, need h/w support
from processor.  For example, instructions like,

1. compare and swap: e.g., compare values of two registers and swap if R1>R2
2. test and set: test if value of register R is non-zero and set it to zero
(or the reverse)

In multi-CPU settings, different CPUs have to coordinate when multiple
threads try to grab the same locks, but running on different CPUs.  Often
using "SMP barrier" instructions.

Two basic locks in Linux, built on top of architecture instructions:

1. spinlock: a busy, exclusive lock,
- cannot sleep in CS (critical section)
- Consumes CPU cycles, but is a fast lock (just a couple of instrux).
- good when the CS is short/quick, e.g., making changes to memory structures.
- spin_lock(), spin_unlock().

2. mutex: an exclusive lock
- can go to sleep: while "lock owner" (the one who got into the CS) is in
  the CS, all others are waiting on the lock.
- all waiters go to scheduler WAIT state (a wait queue)
- mutex is a slower lock, takes 100s of instructions to un/lock
- intended for longer CSs, perhaps while waiting on I/O
- part of "struct mutex" includes a "struct waitq" of waiters
- when lock owner releases a mutex, one or more waiters from waitq are moved
  to READY state, so they can run next.
- Linux: mutex_lock(), mutex_unlock(), mutex_init().

In exclusive locks, only one can enter the CS at a time.  This isn't good
when most users read a shared d-s, and only occasionally modify (write) it.
(If no one modifies d-s, no need for locks at all.).  When you have many more
readers than writes of a shared d-s, we use a "read-write semaphore"
(rwsem).

rwsem:
1. multiple readers can enter CS at the same time
2. only one writer can enter CS at the same time (no readers allowed)

rwsem could starve readers if too many writers:
1. prioritize older readers waiting over new writers
2. rwsem may be wrong type of lock, consider a plain mutex


//////////////////////////////////////////////////////////////////////////////
// rwsem: multiple readers, one writer
struct rwsem {
  int state; // rwsem state: -1 a writer, 0 no one locking, >0 there's "N" readers
  spinlock L; // protects "state" and QW/QR
  waitq QW; // readers waiting on rwsem
  waitq QR; // writers waiting on rwsem
};
struct rwsem *rwsemp; // assume was kmalloc'd and initialized

rwsem_read_lock()
{
  lock(L);
  if (state > 0) // no one has the lock or at least one reader
    if (empty(QW))
      state++; // new or first reader
    else
      add_me_to_waitq(QR); // add this reader to waitq (no starvation of QW)
  if (state == 0) // no one has the lock or at least one reader
    state++; // new or first reader (XXX)
  if (state < 0) // there's an active writer
    add_me_to_waitq(QR); // add this reader to waitq
  unlock(L);
}
rwsem_read_unlock()
{
  lock(L);
  if (state == 0) // should never happen
    BUG();
  if (state < 0) // active writer: shouldn't happen
    BUG();
  if (state > 0) // active readers
    state--;	 // declaring "I'm done reading"
  if (state == 0) // we dropped the last reader
    if (!empty(QW))
      wakeup_oldest_writer(WQ);
    else if (!empty(QR))
      wakeup_oldest_reader(WR); // or even wakeup some or "all" of them
  unlock(L);
}
rwsem_write_lock()
{
  lock(L);
  if (state == 0) // no one has the lock
    state--; // the one writer
  if (state < 0) // active writer
    add_me_to_waitq(QW); // add this writer to waitq
  if (state > 0) // active readers
    add_me_to_waitq(QW); // add this writer to waitq
  unlock(L);
}
rwsem_write_unlock()
{
  lock(L);
  if (state == 0) // should never happen
    BUG();
  if (state > 0) // should never happen
    BUG();
  if (state < 0) // probably "me"
    state++; // release the lock (now state is 0)
  if (state == 0) // the one writer is done
    if (!empty(QW))
      wakeup_oldest_writer(WQ);
    else if (!empty(QR))
      wakeup_oldest_reader(WR); // or even wakeup some or "all" of them
  unlock(L);
}