Message boards : Number crunching : Waste of resource.
Author | Message |
---|---|
adrianxw Send message Joined: 18 Sep 05 Posts: 653 Credit: 11,840,739 RAC: 274 |
As I understand things, a work unit has "a problem", it generates a random number then uses the program to run the model with that random start point. If it finishes within the time limit set by the user, it generates a new random number and repeats. Thinking about that, setting a long run time may not be a good idea. Rational, the project has estimated the number of runs needed to produce the result they need. They estimate the number of work units required to achieve such a result, (they can estimate this from in house runs on their own machines). They sit back and wait. By setting a long run time, I generate more results per work unit on both of my machines. Others are quite possibly doing the same thing. The number of runs required when the initial estimate was made is reached earlier. Are further work units then not sent, or are they left in the queue to, basically, waste the resource? Wave upon wave of demented avengers march cheerfully out of obscurity into the dream. |
Grant (SSSF) Send message Joined: 28 Mar 20 Posts: 1670 Credit: 17,495,896 RAC: 24,629 |
Are further work units then not sent, or are they left in the queue to, basically, waste the resource?Each Task is different data, so it is worth doing all the other Tasks. The default from the project is 8 hours as that is what they have determined gives them a useful result from the available hardware. If running a Task for longer makes you happy, then no problem, the extra work is still of benefit. But it's not necessary. However the fact that your system is over committed and it takes you over 16 hours to do 12 hours of actual processing really is just a waste of your computing time & power. 4 hours of computing time that's not used to actually compute anything is a waste of of resources IMHO. Grant Darwin NT |
Bryn Mawr Send message Joined: 26 Dec 18 Posts: 389 Credit: 12,069,196 RAC: 14,319 |
As I understand things, a work unit has "a problem", it generates a random number then uses the program to run the model with that random start point. If it finishes within the time limit set by the user, it generates a new random number and repeats. I agree, they set the default for a reason - we should not change it without a better reason. |
Mad_Max Send message Joined: 31 Dec 09 Posts: 209 Credit: 25,777,441 RAC: 14,462 |
This number of "runs" it really big, like from tens of thousands to hundreds of thousands runs per model, in some cases it can be more than a million. The fact that we increase the number of runs in one task by increasing the target computation time does not affect anything - the total number is regulated by the number of computed and validated results on server side. When scientists decide that they have already received enough runs for this particular model/goal, they simply cancel all remaining tasks of the same series that are still in the queue. Including ones that have been already downloaded to volunteer PCs (probably you have already noticed this sometime - when a part of tasks from a series that worked without problems / errors is suddenly canceled by a command from the server - "server abort" in WUs status). P.S And Well, in any case, even if we generate some "excessive" result it will not be a complete loss of resources. Because with this pseudo-random search approach used in R@H, more runs/passes = better (more accurate) result become. The cut-off point (when enough is enough to say) is rather arbitrary. It is usually chosen when the improvement in accuracy is already quite insignificant and it is not practical to continue to spend large amounts of computing power on it. But insignificant doesn't mean they don't exist at all. |
Message boards :
Number crunching :
Waste of resource.
©2024 University of Washington
https://www.bakerlab.org