Initial Commit

Author: brainsucker

.travis.yml

@@ -0,0 +1,19 @@
+dist: trusty
+sudo: false
+language: generic
+branches:
+  only: master
+script:
+- openssl aes-256-cbc -K $encrypted_817a849a5ddb_key -iv $encrypted_817a849a5ddb_iv
+  -in deploy_rsa.enc -out /tmp/deploy_rsa -d
+- eval "$(ssh-agent -s)"
+- chmod 600 /tmp/deploy_rsa
+- ssh-add /tmp/deploy_rsa
+- export COMMITTER_EMAIL="$(git log -1 $TRAVIS_COMMIT --pretty='%cE')"
+- export AUTHOR_NAME="$(git log -1 $TRAVIS_COMMIT --pretty='%aN')"
+- git config --global user.email $COMMITTER_EMAIL
+- git config --global user.name $AUTHOR_NAME
+- export NUM_RETRIES=3
+- export BUILD_TIME=10
+- export CLONE_DIR=site
+- bash retry.sh

charming_chatroom.md

@@ -0,0 +1,176 @@
+---
+layout: doc
+title: "Charming Chatroom"
+---
+
+## Learning Objectives
+
+* Networking Components
+* Building working server code
+* Building working client code
+* Network error handling
+* Read/Write error handling
+
+
+## Goal
+
+The goal of this lab is to help you understand networking components. You will accomplish this by writing a real chatroom program. You are going to write a client which can send/receive data to/from server, a server which can receive data from multiple clients and broadcast these messages to each of the clients, and read/write functions than handle the failures of read and write.  The files you must modify are
+
+* client.c
+* server.c
+* utils.c
+
+The files `client.c` and `server.c` provide an outline of what you are expected to do via references to questions in questions.txt. For example, if you see /*QUESTION 1*/ then question 1 in `questions.txt` should help you understand what to do when filling that part of the code.
+**So be sure to answer the questions in `questions.txt` to begin with as these will help you get started!**
+
+
+## Client
+
+The file `chat_window.c` is the control center for setting up the ncurses windows (that's a thing).  You do not need to worry about this, but feel free to look at it if you are interested.
+
+The client executable will accept up to four arguments:
+```
+./client <host> <port> <username> [filename]
+```
+
+* `host`     - The address the client should connect to.
+* `port`     - The port number to connect to the host on.
+* `username` - The name you want to be displayed in the chatroom.
+* `filename` - Optional argument - will disable ncurses and write bytes received from the server to the output file.
+
+The client performs two main functions: (1) writes user input to the server and (2) writes bytes from server to user.  We have handled the overall logic of reading from user, writing to server, reading from server, and writing to user.  Your job is to set up the client and connect it to the server.
+
+Implement the provided function to use a TCP IPv4 connection and connect to the host at the given port.  A signal interrupt will be sent to the client as a flag to tell your client to exit. To be precise, when your client program receives a `SIGINT` it should free memory, close sockets, and gracefully exit the program.
+
+**Notice** the writing and reading to the server use `write_all_to_socket()` and `read_all_from_socket()`.  You will have to implement these functions to handle the failures of read/write calls, but more on that later.
+
+The figure below gives you an idea about how the client side architecture looks like: ![Alt](/images/ClientArch.png "Title")
+
+So to sum up, your job in the client program is:
+
+* Implement the running client and closing client functions
+* Set up the network connection (TCP + IPv4).
+* Launch threads to read from the server.
+* Launch threads to write to server.
+* Free memory you allocate.
+
+**Note:** You do not need to modify any of the code in `client.c` except for the function `connect_to_server()` and `close_server_connection()` in order to get the client successfully working.  However, you *may* modify any of the other code if you want, but **be careful**.
+
+
+## Server
+
+```
+./server <port>
+```
+* port - The port number to accept connections on.
+
+Similar to `client.c`, a lot of the functionality in `server.c` has been implemented for you.  Your job is to set up the server to use TCP IPv4 with reusable ports and gracefully close the server when `SIGINT` is received.
+ The figure below illustrates how a message propagates through the system: ![Alt](/images/MessageBroadcast.png "Title")
+
+To sum up, you have to:
+
+* Implement `run_server()` and `close_server()` (the signal handler for SIGINT)
+* Set up connections (TCP & IPv4).
+* (There is a giant while-loop - you need to do something in it)
+
+Here is the overall client-server architecture:
+
+![Alt](/images/OverallArchitecture.png "Title")
+
+## Read/Write Failures
+
+Read and Write calls (general `read`/`write` - this extends to `recv`, `send`, etc.) can fail to send/receive all bytes.  Here is the pseudocode for read all to socket.
+
+```
+while number of bytes is not the number needed:
+    return_code = read bytes from socket
+    if return_code = 0:
+        return bytes read
+    else if return_code > 0:
+        add return_code bytes to counter
+    else if return_code == -1 and error was interrupted:
+        try again
+    else:
+        return bytes read
+return bytes read
+```
+
+
+In `utils.c/h` we have declared the functions `read_all_from_socket` and `write_all_to_socket`.  You need to implement these functions to read/write from/to a socket and handle the failures of read/write (defined above).  You should look at `utils.h` for detailed info on what your functions should do.
+
+Messages in our server/client will be exchanged in the following format:
+
+```
+<message_size><message>
+informally: 0x0000000C"hello world\n"
+```
+where the **first 4 bytes** of the message indicate the size of the message in bytes.
+We use network-to-host byte-order (ntohl) and host-to-network byte-order (htonl) for this.
+We've given you `get_message_size()`, you **must** implement `write_message_size()`.  As we will be testing your server and client separately, and because our server and client expect this setup, be sure this works; it is like our "protocol".
+Please see how `get_message_size()` it is done in `utils.c`
+
+In `user_hooks.c` we have given an example of how to emulate a read failure (feel free to modify this file by adding a write failure call).  You can include `user_hooks.h` in your utils/server/client code to test your error handling.
+**BE SURE NOT TO LEAVE ANYTHING RELATING TO `user_hooks` IN YOUR FILES.  THEY ARE THERE FOR TESTING, THE AG WILL NOT COMPILE THEM**
+
+## Error Checking
+
+### Networks Break!
+
+Many things can go wrong when working with networks. Be sure to do plenty of error checking! If anything fails, print the error message and **exit(1)**. For each networking function call (except one of them) that you will be using, use **perror(NULL)**. For the one that you cannot use `perror`, use the man pages to learn how to print the error. You must figure out which function does not set `errno` upon failure and thus you cannot use `perror`.
+
+
+## Testing
+You should implement tests for testing functionality of client and server side code separately. That way you don't worry about client-server interactions.
+You can then consider cases where a message is written to one client, is sent to the server and is then broadcasted to all the other clients running. To test such a code path, you start a test that writes to one of the clients **c** and then verifies whether all the clients have received the message that was provided as input to **c**.
+
+Note, you are writing a server that could potentially run for a long time. This is exactly a scenario where memory leaks can be very costly and cause your server to crash. So ensure there are no memory leaks in your implementation.
+
+Otherwise, we will leave it open ended to let you come up with other interesting test cases.
+
+### We provided you with a Makefile.
+
+You can run the client and server as follows:
+```
+./client <address> <port> <username> [filename]
+./server <port>
+```
+
+
+Test your server and client with a friend! On your VMs you can connect to each others' machines. Just ask them what their machine number is!
+
+
+```
+ ./server 12345
+Waiting for connection...
+Connection made: client_fd=4
+^CEnding Server
+```
+
+```
+ ./client {{site.semester}}-cs241-XYZ.cs.illinois.edu 12345 steve
+steve: hello there!
+alec: sending bytes
+^CClosing Client
+```
+Notice the **XYZ**, that is the machine number you will use to connect to the server (the person hosting's machine)
+In the above "terminals", there are things like "Waiting for connection..." and "Ending Server": do not worry about having to do this. It is simply flavor text printed to the window. Feel free to do that; we will only test the bytes that are sent and received.
+**Because it's particularly important for this lab, we want to reiterate that you should not have any memory leaks :)**
+
+## Grading
+
+*   Client must function as specified
+*   Server must function as specified
+*   Implementation of `read_all_from_socket` and `write_all_from_socket` to handle failures
+*   Errors must be handled as specified
+*   No memory leaks
+
+Once again, you do not need to modify any of the provided code in order to complete the lab, but you *can* if you want (however, be very careful; we are not responsible for your changes )
+
+
+## Interesting Things
+
+We are using `pthread_cancel`, `pthread_detach`,  `pthread_cleanup`, and `ncurses`.  It is not necessary to know about these (actually *cancel* might have been on one of your quizzes), so don't let them distract/confuse you.  However, if you are interested, definitely read up on them.
+
+In the Makefile, we have `-I./includes`.  This lets you do `#include "something.h"` instead of `#include "./includes/something.h"`
+
+Also `#pragma once` in the .h files

cleanup.sh

@@ -0,0 +1,3 @@
+#!/bin/bash
+
+rm -rf ${CLONE_DIR}

critical_concurrency.md

@@ -0,0 +1,105 @@
+---
+layout: doc
+title: "Critical Concurrency"
+learning_objectives:
+  - Synchronization Primitives
+  - Common Patterns in Multi-Threaded Programs
+  - Thread-Safe Datastructures and Their Design
+wikibook:
+  - "Synchronization, Part 1: Mutex Locks"
+  - "Synchronization, Part 2: Counting Semaphores"
+  - "Synchronization, Part 3: Working with Mutexes And Semaphores"
+  - "Synchronization, Part 4: The Critical Section Problem"
+  - "Synchronization, Part 5: Condition Variables"
+  - "Synchronization, Part 6: Implementing a barrier"
+  - "Deadlock, Part 1: Resource Allocation Graph"
+  - "Deadlock, Part 2: Deadlock Conditions"
+---
+
+## Overview
+
+There are four main components to this lab, three of which are graded. These are Rendezvous (not graded), Semamore, Barrier, and Thread-safe Queue. Each of these represent very common synchronization problem (or slight twists on them) that will do you well to become familiar with.
+
+Good luck!
+
+## Rendezvous (UNGRADED)
+
+This is a problem for you to think about. We have provided a worked solution to this problem but PLEASE try to solve this problem before looking at the solution!
+
+Problem description:
+
+Given two threads, `a` and `b`, and the fact that both have to run two tasks (`a1`, `a2`, `b1`, `b2`), how do you get both `a1` and b1 to run before either `a2` and `b2`? In `rendezvous.c`, you need to modify the two functions (`modifyB_printA` & `modifyA_printB`) using semaphores so that both quotes `A` and `B` are modified before being printed.
+
+
+## semamore.c
+NOTE: A semamore is **NOT** a real thing! It is simply a made up clever name! This means you can't use them in future assignments (unless you re-implement it).
+
+A normal semaphore blocks when the value within the semaphore reaches 0. A semamore blocks when it reaches 0, but also blocks when it reaches some maximum value. You can think of a semamore as a top-bounded semaphore. In semamore.c, you are given four functions to work on, `semm_init`, `semm_wait`, `semm_post`, `semm_destroy`. `semm_post` is the important difference. When `semm_post` reaches `max_val` (as defined in the semamore struct in semamore.h), it blocks.
+
+There are four functions in total you will be writing:
+
+* `void semm_init(Semamore *s, int value, int max_val);`
+* `void semm_wait(Semamore *s);`
+* `void semm_post(Semamore *s);`
+* `void semm_destroy(Semamore *s);`
+
+## barrier.c
+
+In rendezvous you saw an example of an one-time-use barrier.  Now, you get to build code to support a reusable barrier.  At the cost of being redundant, a reusable barrier is one that can get used more than once.  Say you have threads executing code in a for loop and you want them to stay in sync.  That is, each thread should be on the i'th iteration of the loop when every other thread is on the i'th iteration.  With a reusable barrier, you can stop threads from going to the i+1'th iteration until all of them have finished the i'th.
+
+Note that most barrier implementations (including the pthread library barrier) are "reusable", but never say so.  This is because it simply does not make sense to have a "not-reusable" barrier.  Thus, we are only iterating to you that the barrier your build should be reusable so that you understand what  it means.
+
+You can find more info in the [barriers coursebook entry](http://cs241.cs.illinois.edu/coursebook/Synchronization#barriers)
+
+Your goal is to implement the functions
+
+* `int barrier_destroy(barrier_t *barrier);`
+* `int barrier_init(barrier_t *barrier, unsigned num_threads);`
+* `int barrier_wait(barrier_t *barrier);`
+
+so that a `barrier_t` using these functions is a working reusable barrier.
+
+## queue.c
+
+**NOTE: Do not use semaphores or your semamore here.**
+
+Your task is to build a thread safe queue, that also may or may not be bounded, by implementing the functions in queue.c. The maxSize of the queue can be set to either a positive number or a non-positive number. If positive, your queue will block if the user tries to push when the queue is full. If not positive, your queue should never block upon a push (the queue does not have a max size). If your queue is empty then you should block on a pull. You should make use of the node struct to store and retrieve information. In the end, your queue implementation should be able to handle concurrent calls from multiple threads. `queue_create` and `queue_destroy` will not be called by multiple threads.
+
+The queue is completely independent of the data that the user feeds it. The queue should make use of the constructors and destructors provided by the user to handle data.
+
+Your goal is to implement the functions
+
+* `queue* queue_create (ssize_t max_size);`
+* `void queue_destroy (queue* this);`
+* `void queue_push (queue* this, void* element);`
+* `void* queue_pull (queue* this);`
+
+## Testing
+
+**Testing is ungraded, but highly recommended**
+
+Since the implementation of your semamore is quite close to an actual semaphore, please test this on your own in a variety of ways. Be careful of race conditions! They can be hard to find!  We've given you a `semamore_tests.c` file to write tests in.
+
+
+For `barrier_test.c` we have provided you with a simple test case.  Feel free to expand on it, as it is not exhaustive/perfect.  Learning how to use the barrier is just as important as writing it, since you will be using barriers on the Password Cracker MP :)
+
+For `queue_test.c` we would like you to write tests yourself.  Learning to write tests for multi-threaded code is very important.  You will also be using this queue in the Password Cracker MP :)  (we will give you a working version; you will not be penalized on the MP for not successfully completing the lab)
+
+### Thread Sanitizer
+
+We have another target executed by typing `make tsan`. This compiles your code with Thread Sanitizer.
+
+ThreadSantizer is a race condition detection tool. See [this page](http://cs241.cs.illinois.edu/coursebook/Background#tsan) for more information.
+
+**We will be using ThreadSanitizer to grade your code! If the autograder detects a data race, you won't automatically get 0 points, but a few points will be deducted.**
+
+
+## Helpful Hints and Notes
+
+*   Make sure you thoroughly test your code! Race conditions can be hard to spot!
+*   Attempting to visualize your code or diagram it in certain cases can sometimes be a huge aid and is highly recommended!
+
+** In any of `semamore.c`, `barrier.c`, or `queue.c` you may not use semaphores or pthread_barriers **
+
+**ANYTHING not specified in these docs is considered undefined behavior and we will not test it**
+For example, calling queue_push(NULL, NULL) can do whatever you want it to. We will not test it.