Principles and Practice of Parallel Programming
COMS 4130, Fall 2012

(Administrative) (Syllabus) (Project Milestones) (Project Suggestions) (X10 Code Examples) (Other Resources) (Piazza)

Administrative

Lectures

Lectures will be held on MW 2:40-3:55 pm in 227 MUDD

Course Staff

Email Office Hours
Vijay Saraswat vs2345@columbia.edu MW 1:40-2:40, 468 CSB
Martha Kim martha@cs.columbia.edu Tu 11:30-12:30, Th 2:30-3:30, 469 CSB
Ivan Wu zw2229@columbia.edu F 2-4, TA Room, 1st Floor Mudd

Course Overview

Learning how to program parallel computers (multi-core, clusters) productively and efficiently is a critical skill in this era of concurrency. The course will provide an introduction to modern parallel systems and their performance characteristics. It will cover the fundamentals of data-structure design, analysis and implementation for efficient parallel execution; programming abstractions for concurrency; and techniques for reasoning about the behavior and performance of parallel programs. Particular topics to be covered include: data parallelism, fine-grained concurrency, locality, load-balancing, overlapping computation with communication, reasoning about deadlock-freedom, determinacy, safe parallelization, implementing frameworks for concurrency (such as Hadoop Map/Reduce), debugging for correctness and performance. Students will study many parallel programs drawn from a variety of application domains (including high-performance computing, large-scale graph analyses, machine learning, game playing) Students will be expected to complete a series of parallel programming homework assignments. The center-piece of the course will be a semester-long team project focussed on implementing a challenging programming problem with good performance on a cluster of multi-cores, using a modern parallel language, X10.

Prerequisites

Experience in Java, basic understanding of analysis of algorithms. COMS W1004 and COMS W3137 (or equivalent).

Late Policy

All homeworks are due by 11:59PM on their duedate. You have one late day to use throughout the semester. You may use it on the homework of your choosing. No other extensions will be granted.

Collaboration Policy

We take academic honesty extremely seriously, and expect the same of you. In this course, you are free to discuss homework problems with your classmates. However, each student is to write up his or her own solution and is expected to be able to explain and reproduce the work she or she submits. Please note the names of your collaborators at the top of your homework submission. Apart from these exceptions, the Computer Science Department's Academic Honesty policy is in effect.

Grading Formula

Homeworks: 30%
Quizzes: 35%
Project: 35%

Syllabus

Date Unit Topic(s) Notes Quiz Homework Milestone
Sep 5 Unit 1: X10 Overview; Begin X10
Sep 10 Unit 1: X10 Serial X10; APGAS
Sep 12 Unit 1: X10 APGAS cont.
Sep 17 Unit 2: Hardware Background; Retrospective HW 1 (solutions)
Sep 19 -- Challenge Intro
    Sep 24 Unit 2: Hardware Hardware case studies Quiz 1
    Sep 26 Unit 3: Perf Analysis Language Model Milestone 1: Project Proposal
    Oct 1 Unit 3: Perf Analysis Scaling Theory HW 2 (solutions)
    Oct 3 Unit 3: Perf Analysis Perf Measurement/Debugging
    Oct 8 Unit 3: Perf Analysis Language Model (cont.)
    Oct 10 Unit 4: Safe X10 Intro to Safety
      Oct 15 Unit 4: Safe X10 Structural Operational Semantics
      Oct 17 Unit 4: Safe X10 Structural Operational Semantics (cont) Quiz 2 HW 3 (solutions)
      Oct 22 Unit 4: Safe X10 Structural Operational Semantics (cont) Milestone 2: Performant Serial Implementation
      Oct 24 Unit 5: Semantics of Concurrency TBA
        Oct 29 Unit 5: Semantics of Concurrency TBA
          Oct 31
          Cancelled (Sandy)
          Nov 5
          Academic Holiday
          Nov 7 Unit 6: Indeterminacy Parallel Prefix; Indet Intro
          Nov 12 Unit 6: Indeterminacy Lock-Free Queue Quiz 3
          Nov 14 Unit 6: Indeterminacy Queue (cont); Peterson's Lock HW 4
          Nov 19 Unit 6: Indeterminacy Bakery Lock Milestone 3: Functional Concurrent Implementation
          Nov 21 Unit 6: Indeterminacy Puzzles
          Nov 22
          Thanksgiving Holiday
          Nov 26 Unit 7: Conditional Atomicity Blocking Synchronization
          Nov 28 Unit 7: Conditional Atomicity Clocks
          Dec 3 Unit 7: Conditional Atomicity Streams
          Dec 5 Unit 7: Conditional Atomicity Hadoop Quiz 4
          Dec 10 Unit 7: Conditional Atomicity Streams HW 5/6
          Dec 17 Project Presentations

          Project Milestones

          Milestone 1: Project Proposal

          By this date you should have Formed groups: You may group yourselves in duos or trios. By default we expect everyone in a group to contribute equally and thus by default each team will share a project grade. However if it becomes clear that the contributions are grossly uneven across a group we may adjust the scoring on an individual basis. Select a project topic. By the end of the semester your goal will be to implement a scalable (across multiple places) parallel implementation of your selected topic. At this milestone you should be asking yourselves the following questions: Do I understand this problem? Is this problem data or compute intensive? Is there sufficient work to parallelize across multiple workers and places? Look for a "Project Topics" note on Piazza if you'd like some ideas to get you started. Be talking to Vijay, Martha and the TAs prior to this milestone to help you refine your plans. Write up a 1 page document listing your teammates and describing the problem you've selected. Email this document to both TAs.

          Milestone 2: Performant Serial Implementation

          By this date you should have implemented a performant serial version of your project code. For this milestone you will submit your code (readable and commented), a makefile, test programs that test the functionality of your code, programs that test performance of your code, any files needed to run functionality and performance tests. Your make file should be able to run your performance and functionality tests so that all we need to do is run "make perftests" or "make functests" and see the results.

          Milestone 3: Functional Concurrent Implementation

          By this date you should have implemented a functional concurrent implementation of your code. You should submit your code (as in Milestone 2), a 1 page design document outlining your parallelization strategy, and a report describing how the performance of your code scales. (It is not essential that your code scale *well* at this stage, but you should begin measuring performance at this stage in order to improve the performance in subsequent weeks.)

          Milestone 4: Final Reports and Presentations

          Each group will have ~30mins to present their projects to the course staff. You should plan on a brief 10 min presentation followed by 15 mins of questions and discussion. At this time you will turn in your code (commented, with makefiles and inputs as usual), a single page design document and a report outlining the performance of your code.

          Project Suggestions

          X10 Code Examples

          Other Resources