Moed A, Q 5
|Those who marked answer "D" to question 5 and got their grade reduced - please submit an appeal by e-mail to Mark with the subject: |
CDP Reception houlrs before the exam
|For Moed Bet attendees, reception hours by Mark will be held at 9.00-10.00, on Monday, 15/03, in Fishbah 453 (my new office)|
Here's how you get there - (those who built this place apparently used the original plan of Minotaur's labyrinth of Crete to keep students away from their teachers)
Go to Fishbach building ( between Ullman and Hashmal ), turn right, proceed to the end of the corridor, then go upstairs 1 floor, keep straight to the end of the hall, and then turn right. You'll see a corridor, my room is the third on the left.
MPI question, Exam A
|Those who used IProbe in the answer to the MPI question please submit the formal appeal.|
Mark's reception hours before exam - take 2
|Due to multiple requests - Tuesday, 12.30, Taub 4.|
Mark's reception hours before exam
|The reception hours will take place in Taub 3, 12.30, Sunday|
No material is allowed at the exam
|For those who could have misunderstood - no material is allowed whatsoever, only your pen and your head.|
Reception hours before exam + material
|1. No lecture/tutorial material is allowed |
2. Exam will be 2/3 multiple-choice (american) 1/3 open questions
3. Material included - all the things taught at the lectures and tutorials excluding the last OpenCL lecture. Those who missed the lecture/tutorials should consult their peers re/ what was covered from the slides, and not the course staff.
Reception hours before the exam:
Prof. Schuster - Thursday, 11/2, 13.00 - only questions re/ the lectures
Mark - Sunday 14/2, 12.30 - everything excluding the lectures on transaction memory and weak operations
HW3 and HW4 appeals
|Appeals for HW3 and HW4 will be held in Nadav's room at 325 on Tuesday 9/2 16:00-17:00 and on Wednesday 10/2 16:00-17:00. |
Appeals can also be sent by e-mail to namit /at/ cs and are recommended in case points were deducted due to compilation errors.
Problem running BSP
|It seems that the course is having trouble with the BSP implementation on SUN. So, here are the guidelines how to survive and let other run the tests as well:|
1. limit the maximum number of processes invoked to 4
2. use command "ps -o user -a -A -o pid -o args | grep <your user>" to see how many your old processes are running, or without grep, who's running anything
3. kill the dangling processes
4. use "ipcs" to make sure that you're not holding sempahors
5. use ipcclean to remove all those that you're still holding
I have no other means to restrict the system usage, so I rely on your sense of understanding that you're not alone and others should complete the hw as well
NO MAILS WILL BE ANSWERED TODAY ANYMORE.
Problem when running BSP
|In some cases BSP fails to clean the state after failure, such as if your program crashes during the debugging.|
The garbage is accumulated, and at some point BSP refuses to start with an error "not enough semaphores" or something like that.
In such a case, simply invoke "ipcclean", just as recommended in the error message, and that will do the job
|HW4 is postponed until 24Jan, 23.55|
Reminder - schedule for the last week
|Sunday - last lecture - this will be the summarizing tutorial for both groups|
Sunday - no tutorial at 15.30
Wednesday - last tutorial on Condor DAGman
Last lecture - Wed, 13/1, 12.30-14.30, Taub 9
|The very last lecture of this semester will be held on 13/1 by Ofer Rosenberg from Intel on the next generation programming standard OpenCL for GPUs.|
Ofer was not available on Sundays, that is why we were forced to schedule it in "hahafsaka hapeila".
So Taub 9, 13/1, 12.30-14.30
The lecture time (17/1) will be used to go over the questions from the previous exams.
HW4 is out
|Due 20/10. The earlier you submit, the better for you...|
HW2 - competition results.
|HW2 was graded and the results of the competition are here.|
The first two rows show the clear winners of the first and the second place, entitling for 25 and 20 points bonus respectively.
032698151-039715578 0.985747 1.93483 3.807786 6.647558 11.887939
200682698-321440265 1.023423 2.007715 3.948048 6.066763 8.553888
Unfortunately the third place is not awarded to anyone since it is shared among ~8 pairs..
The grades will be published shortly.
The winners are required to claim the bonus - approach me after the tutorial/lecture and be ready to explain your solution.
Problem with using derived data types and Scatter
|By now I guess most of you tried to use derived datatypes, and it all seemed working well until you tried sending them with MPI_Scatter/Gather and alike.|
It works with Broadcast, but doesn't work with scatter/gather - why??
This was actually a challenge for you to discover, but I realize it was a bit too tough.
The derived datatypes in MPI are created in a way that the data from the original, C datatype, fits in.
However nobody said that they will not contain any additional information, or, will not be aligned. The total size of memory
used by MPI datatype is called datatype extent. So this extent may be larger than sizeof(C datatype).
So as long as you send only one element of the datatype (count=1) - everything is fine.
But when you send 2 elements from the contiguous region, or array like with MPI_Scatter, or MPI_Send with count>1, MPI would send
extent*count bytes from your application buffer, and not sizeof(C datatype)*count, which is what you have in your array.
For example: in your data type DT you have 2 ints and float -> so it's total 12 bytes. But MPI extent for the MPI datatype MPI_DT might be, say, 16 bytes.
So with the array with two elements like this:
executing MPI_Send(arr,2,MPI_DT ... ) ,
will result in MPI sending 32 bytes from arr, instead of 24! -> this would be a segmentation fault.
For MPI_Scatter it would send garbage to the other processes.
So how to cope with that? We can actually force MPI to create the datatype having the right size, in order for such datatype
to be sent via Scatter and alike.
For that we use marker datatypes LB and UB (lower bound and upper bound) surrounding the actual datatypes of our structure.
I uploaded the example of the datatype definition which does that correctly, and works on ds-fire0.
You can download it from the Tutorials, MPIII section.
time measurements, and questions that won't be answered anymore
|1. Time measurement: use UNIX "time" utility.|
2. I'm starting to get REALLY tired, to say the least, of the questions in the form of:
+"how do we implement X/run mpi/compile our program"
+"my program does not work, have a look at the code, help!"
+"I have no idea of how to... , give me a hint"
The generic answers are
1. USE ANY WEB SEARCH ENGINE, since you definitely have access to the internet if I keep getting these "sophisticated" questions from the soon-to-be software engineers.
2. I will not look at the code, since there are too many of you. And, see the first generic answer too..
3. would your ask your manager at Intel/MS/whatever to give you hints? It will be the last day of your employment.. Same here.
I want to clarify the rules of the game:
I will answer the questions only if
1. I fill the person really tried hard to find the answer and failed
2. It is formulated in a proper, polite way
3. It exposes misunderstanding of the homework goals
4. It is a tricky one that I may have no answer to but I'll do my best.
Spam will be trashed and the spammers punished.
does not work properly since there is a stupid mistake in the make file. I bet most of you figured out how to fix that - simply fix the make target "tsp_static" to produce "tsp_static" executable, and not "tsp"
DS-FIRE0 is back online
|DS-FIRE0 resurrected from dead. You've got 5 days of postponement, which brings it to the 10th of Jan, 23.55|
Hope it won't die again
Use of collective communications, HW3 Part 1
|To simplify the solution you can use collective communication calls twice in the beginning of the run and once in the end.|
Failure of ds-fire0
|It turns out that something happened with ds-fire0 server. ( you guys managed to kill even most reliable bullet-proof SUN server )|
Unfortunately currently nobody can connect (including me), and fix the problem, and the physical access to the room where the server is located will be possible only on Sunday.
So, those who didn't develop on Windows, will have to wait until then. Postponement will be given by the number of days the server was down.
Those who celebrate New Year may now relax and enjoy the party ;-)
Today lecture - Condor
|Today the lecture will be on Condor. MapReduce lecture will be given on the 3rd of January.|
Sorry for the last minute change
Clarification re/ memory complexity requirements for HW3
|The requirement of O(1) memory complexity with respect to # MPI processes is intended only for the dynamic solution.|
It refers to the memory complexity of a single MPI process and not all processes together.
That is, as long as one MPI process does not allocate an array of size of # MPI processes - you're doing fine.
This requirement is the example of a constraint to allow scalability to many processors.
|Appealing regarding HW1 can be done at Thursday, 24/12, 16:30-17:30 and at Monday 28/12, 16:30-17:30 - on Nadav's room (325).|
Please bring hard-copy of your graded submission with you.
How to open the archive on ds-fire0
|Note the change in the way the archive should be open:|
/usr/local/bin/tar zxvf tsp-sun.tar.gz
( instead of tar zxvf tsp-sun.tart.gz)
Wednesday - Day off
|For all those starving for knowledge despite Hanuka... |
Wednesday is the day off, so there's no tutorial.
Happy Hanuka and/or pleasant solving of HW3 ( hope these are not mutually exclusive )
HW3 is out
Note - the HW is hard and requires a lot of thinking although it does not look like that from the first sight.
HW2 delayed to 13/12, 23.55
|Thanks to your semester representative, the deadline of the HW2 is moved to 13/12, 23.55|
No further postponements whatsoever.
Tutorials schedule announcements
|As you know Mark is in miluim this week, so the tutorials on MPI-2 will be held by Nadav.|
Sunday next week (13/12) there will be a complimentary tutorial on MPI-3 for the Wednesday group at 16.30, the room will be announced later.
Problem with time estimate
|There's been a problem with the time estimation of your run. |
The new version fixing that problem should be re-downloaded from the website.
HW2 is out
|You'll get details about login later on|
deadline - 11/12
User distribution for hw 2,3,4
|In order to allow you to login to the server for developing and running your HW2,3,4, each COUPLE |
should send me ( ) email with the subject (EXACTLY this one - all others will be ignored):
CDP password request
And in the mail body - your IDs and names.
Again, only one per couple. The passwords and all the details will be sent later.
I should get this mail not later than this Tuesday, 1 Dec.
Apologies for the missed tutorial on Wed
|Friends, there was some misunderstanding between me and Nadav, and the tutorial was not held. |
We apologize for this. He will hold the tutorial next week. Note, the first group ( Sunday) will have no tutorial next week
HW1 - 2 days extension
|I realize, from the e-mail traffic I get lately, you guys are working hard to be on time.|
I know the HW has been harder than you thought, and it's on purpose - as I said the first and the third are substantially harder than the second and the fourth.
Because of the flu waves hitting us here and all these moadey bet/gimel/daled and etc..., take additional 2 days and have a good sleep tonight..
Those who got extension from me and it's less or equal to 2 days - it is canceled.
The electronic submission is being extended.
Memory problems in HW1
|I am getting many questions re/ "OutOfMemory" in java when running with many threads. |
If your asymptotic complexity is indeed O(field size) and does not depend on the number of generations or number of threads, the problem is less critical.
We will try your code on a field 20x20 with 20x20 split, and 100x100 field with 10x10 split.
We will run it with 1.5GB of heap size: we will invoke your code as follows -
java -Xmx1500M <your program>
If it works for you, it will work for us too.
|1. Next week the tutorials will be on MPI ( tutorial #1) held by Nadav Amit. There will be no tutorial on the week after that (22/11, 25/11) |
2. The lecture next week will be on OpenMP by Nadav Amit. The week after the lecture will be on Thread Building Blocks.
Last but not least - please check the FAQ on the hw regularly.
The following is very important so I publish it now:
1. Any kind of busy wait in the Ex1 is not allowed and is wrong. Namely, if your thread goes over its part of the field and if it finds nothing it continues in an infinite loop until the neighbors were updated so it can continue - that's wrong, since this way when many threads are invoked - many more than #CPUs, your program will be very slow.
2. You should assume that ALWAYS all 8 neighbors are required to compute the next state.
|Complementary tutorial on Optimistic design will be given on Wednesday, 11/11 at 12.30 in Taub 5.|
All other complementary tutorials ( one tutorial is planned for the Wed. group ) will be given on Sundays 16.30
Fix to Hw1
|Please re-download ex1.zip - the was an error in the implementation which resulted in a person to always remain healthy|
Grading policy finalized + HW1
|Two bad news:|
The final (indisputable) grading policy this semester is 25% HW, 75% Exam.
HW1 is ready, due 17th of Nov, 18.00... Note - the HW is different from the one in the previous semesters...
Moed A - the date is fixed
We are in the midst of the intensive efforts to recruit another full time TA, which means that the grading policy may change toward increasing the weight of the HWs. Thus the grading policy info on the course website, as published and announced during the tutorials, may become obsolete.
If this is the factor determining whether to take/not to take the course - wait with your decision until we finalize the grading policy.
Sorry for the mess,
Tutorial 14/10 - canceled
|The tutorial on 14/10 9.30 is canceled and will be complemented later.|