New generators

Message boards : Code and Servers : New generators
Message board moderation

To post messages, you must log in.

1 · 2 · 3 · Next

AuthorMessage
Tomáš Brada
Project administrator
Volunteer developer
Avatar

Send message
Joined: 3 Feb 19
Posts: 632
Credit: 445,023
RAC: 12
Message 3135 - Posted: 25 Feb 2019, 13:36:02 UTC

So, I received source code for new generators. This means a new applications needs to be built. Luckily, the form is the same, so I can just swap the old generators with new and create a padls3 boinc application. The project queue is currently dry, but there are 3319 rows waiting for this app.
ID: 3135 · Rating: 0 · rate: Rate + / Rate - Report as offensive
fzs600

Send message
Joined: 16 Feb 19
Posts: 8
Credit: 2,049,304
RAC: 482
Message 3136 - Posted: 25 Feb 2019, 16:16:00 UTC - in response to Message 3135.  

So, I received source code for new generators. This means a new applications needs to be built. Luckily, the form is the same, so I can just swap the old generators with new and create a padls3 boinc application. The project queue is currently dry, but there are 3319 rows waiting for this app.

the new application has a quorum of 128
is this normal?


nom padls3_24973586bi
applications PADLS Rule 58
créé 25 Feb 2019, 15:38:44 UTC
quorum minimum 128
réplication initiale 128
nombre maximum de tâches en erreur/totales/succès 3, 256, 130
ID: 3136 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Tomáš Brada
Project administrator
Volunteer developer
Avatar

Send message
Joined: 3 Feb 19
Posts: 632
Credit: 445,023
RAC: 12
Message 3137 - Posted: 25 Feb 2019, 16:20:40 UTC

I built first version of padls3. This is for x86_64 Linux and finally also statically linked. I sent a first test row that I have a know-good result already. In order to test reliability on as many computers, first row has very high quorum. This is intentional. Real tasks will have different parameters. Eventually, windows and armv7l app will be built.
ID: 3137 · Rating: 0 · rate: Rate + / Rate - Report as offensive
frankhagen

Send message
Joined: 17 Feb 19
Posts: 4
Credit: 10,089
RAC: 0
Message 3138 - Posted: 25 Feb 2019, 16:57:26 UTC - in response to Message 3136.  

réplication initiale 128
nombre maximum de tâches en erreur/totales/succès 3, 256, 130


ROTFLASTC!!!
ID: 3138 · Rating: 0 · rate: Rate + / Rate - Report as offensive
LCB001

Send message
Joined: 16 Feb 19
Posts: 4
Credit: 5,639,841
RAC: 3,582
Message 3139 - Posted: 25 Feb 2019, 18:44:47 UTC

According to the server status page there are only 33 users with 104 computers currently running this project and not all of those rigs will be running Linux.

Interested in how you are going to get 128 distinct validations from them or am I missing something...
ID: 3139 · Rating: 0 · rate: Rate + / Rate - Report as offensive
frankhagen

Send message
Joined: 17 Feb 19
Posts: 4
Credit: 10,089
RAC: 0
Message 3140 - Posted: 25 Feb 2019, 19:09:26 UTC - in response to Message 3139.  
Last modified: 25 Feb 2019, 19:10:26 UTC

According to the server status page there are only 33 users with 104 computers currently running this project and not all of those rigs will be running Linux.

Interested in how you are going to get 128 distinct validations from them or am I missing something...


that's why if find this funny - sort of...

... well maybe next year.. :)
ID: 3140 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Natalia Makarova
Project scientist
Avatar

Send message
Joined: 8 Feb 19
Posts: 356
Credit: 0
RAC: 0
Message 3141 - Posted: 25 Feb 2019, 19:21:54 UTC

Tomas Brada
I do not understand…
Should each task be processed 128 times?

In my opinion, it is enough to establish a quorum of 2.
ID: 3141 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Tomáš Brada
Project administrator
Volunteer developer
Avatar

Send message
Joined: 3 Feb 19
Posts: 632
Credit: 445,023
RAC: 12
Message 3142 - Posted: 25 Feb 2019, 19:47:15 UTC

In order to test reliability on as many computers, first row has very high quorum. This is intentional. Real tasks will have quorum 1. This is a test of the new build. While it says that it needs 128 quorum now, it does not mean anything. It has been set so the transitioner does what I want. I can see the results immediately and I see that everything is right. The test will be over soon.
ID: 3142 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Tomáš Brada
Project administrator
Volunteer developer
Avatar

Send message
Joined: 3 Feb 19
Posts: 632
Credit: 445,023
RAC: 12
Message 3143 - Posted: 25 Feb 2019, 21:13:38 UTC
Last modified: 25 Feb 2019, 21:14:17 UTC

Output is stable and the combined result matches know-good. Will load tasks shortly.
Also, assigned credit.
# PADLS gen1+gen2 2 4 9 7 3 5 8 6
# KF DLK 211703, ODLK 15
# Users: Tomáš Brada, fzs600, Ralfy, LCB001, zombie67 [MM], frankhagen, Buckey, Sphynx, Sergey Kovalchuk, Bok, philip-in-hong kong, [SG-FC] oki
T28kLD6Ja5vyC3uQ39ut7qNQz
TqRCrhW96QBrkK9X5NLS4mKCd
1h9VXz9ciEGw1LSrZS2cWivFY
RzWXvoY7mJ6TiJGPn7okQs6Bx
3FckeNFVKRTNAzRhR2Wandexv
UF6n3rorF1vrtTpmD12j7Ctz4
SuEy7RG2Q5b3FVqVeM5vUzQT3
SktDcwfGDSpjr1kcLAuYkELsg
9t1KnUgq31Rs58vY7H6aY4Hzv
3DG8aMUAnJNaQZGZx1Vcq1kRV
MFSAwBPwgE3m1VEZ12FRpNxMC
UEQykiN6aH2wHM18n6MBSZAQ7
5vZxDjNQu9oEeVPdBCkiqLmgH
2wJtgnW7BPzvegRGiGYj6vfnG
MJxYNgPe8SMLpc4GD66Rx15gv

The report has wrong title, but content is right.
ID: 3143 · Rating: 0 · rate: Rate + / Rate - Report as offensive
zombie67 [MM]
Avatar

Send message
Joined: 16 Feb 19
Posts: 14
Credit: 1,519,846
RAC: 0
Message 3146 - Posted: 25 Feb 2019, 23:59:47 UTC - in response to Message 3137.  

I built first version of padls3. This is for x86_64 Linux and finally also statically linked. I sent a first test row that I have a know-good result already. In order to test reliability on as many computers, first row has very high quorum. This is intentional. Real tasks will have different parameters. Eventually, windows and armv7l app will be built.


Any chance for a MacOS app?
Reno, Nevada
Team: SETI.USA
ID: 3146 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Natalia Makarova
Project scientist
Avatar

Send message
Joined: 8 Feb 19
Posts: 356
Credit: 0
RAC: 0
Message 3147 - Posted: 26 Feb 2019, 6:33:00 UTC - in response to Message 3143.  

# PADLS gen1+gen2 2 4 9 7 3 5 8 6
# KF DLK 211703, ODLK 15
# Users: Tomáš Brada, fzs600, Ralfy, LCB001, zombie67 [MM], frankhagen, Buckey, Sphynx, Sergey Kovalchuk, Bok, philip-in-hong kong, [SG-FC] oki
T28kLD6Ja5vyC3uQ39ut7qNQz
TqRCrhW96QBrkK9X5NLS4mKCd
1h9VXz9ciEGw1LSrZS2cWivFY
RzWXvoY7mJ6TiJGPn7okQs6Bx
3FckeNFVKRTNAzRhR2Wandexv
UF6n3rorF1vrtTpmD12j7Ctz4
SuEy7RG2Q5b3FVqVeM5vUzQT3
SktDcwfGDSpjr1kcLAuYkELsg
9t1KnUgq31Rs58vY7H6aY4Hzv
3DG8aMUAnJNaQZGZx1Vcq1kRV
MFSAwBPwgE3m1VEZ12FRpNxMC
UEQykiN6aH2wHM18n6MBSZAQ7
5vZxDjNQu9oEeVPdBCkiqLmgH
2wJtgnW7BPzvegRGiGYj6vfnG
MJxYNgPe8SMLpc4GD66Rx15gv

Here are interesting solutions.

Найдено марьяжных КФ:
count[1] = 56
count[2] = 12
count[4] = 1
count[10] = 1
Всего: 70
Найдено соквадратов: 94
КФ соквадратов: 70

Unfortunately, all solutions are not unique.
Repeated solutions will be many.
I wrote
Важное замечание: я не исключила первый генератор в данной ветви эксперимента по той простой причине, что мы с Demis, работая только с первым генератором, пропускали много областей (во всех трёх линейках), в которых были повторения решений.
Иначе говоря, мы проверили далеко не все псевдоассоциативные ДЛК в трёх линейках (для генератора 1).

https://boinc.progger.info/odlk/forum_thread.php?id=107&postid=3161#3161

Rule # 38 was checked only with generator 1, but it was not fully tested, there were a lot of omissions.
Generator 2 is now added and everything is checked again with two generators.
I hope there will be new solutions.

I and XAVER tested this experiment. New solutions found!

All good solutions!
Thank you for your work!
ID: 3147 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Natalia Makarova
Project scientist
Avatar

Send message
Joined: 8 Feb 19
Posts: 356
Credit: 0
RAC: 0
Message 3148 - Posted: 26 Feb 2019, 6:46:18 UTC
Last modified: 26 Feb 2019, 6:47:29 UTC

This is count [10].
Illustration by Demis.
The solution I found in the PADLS experiment in April 2018



Beautiful solution!

This solution is noted in OEIS as the first group of ten ODLS pairs
https://oeis.org/A287695
ID: 3148 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Sergey Kovalchuk

Send message
Joined: 18 Feb 19
Posts: 9
Credit: 69,320
RAC: 18
Message 3149 - Posted: 26 Feb 2019, 10:43:03 UTC - in response to Message 3137.  

first tasks (padls3_24973586) - quorum 12(130), credits 13.68
all subsequent tasks - quorum 1, credits 0

Are they really so irrelevant to the project?
ID: 3149 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Tomáš Brada
Project administrator
Volunteer developer
Avatar

Send message
Joined: 3 Feb 19
Posts: 632
Credit: 445,023
RAC: 12
Message 3150 - Posted: 26 Feb 2019, 10:58:49 UTC - in response to Message 3149.  

first tasks (padls3_24973586) - quorum 12(130), credits 13.68
all subsequent tasks - quorum 1, credits 0
Are they really so irrelevant to the project?

Quoting myself:
In order to test reliability on as many computers, first row has very high quorum. This is intentional. Real tasks will have quorum 1.

This assimilator has, besides from assimilating, new credit awarding algorithm. Credit for completed tasks stays the same, but from today on, credit will not be awarded during validation, but later during assimilation of the batch.

Sergey Kovalchuk: Your results are all right. Taking little too long, but still good. No errors.
ID: 3150 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Tomáš Brada
Project administrator
Volunteer developer
Avatar

Send message
Joined: 3 Feb 19
Posts: 632
Credit: 445,023
RAC: 12
Message 3151 - Posted: 26 Feb 2019, 11:04:36 UTC - in response to Message 3146.  
Last modified: 26 Feb 2019, 12:10:32 UTC

Any chance for a MacOS app?

Yes. There is a chance. Eventually there will be a windows app, armv7l app and darwin app.

Now, please, zombie67, you returned two error tasks: 23764895ii and padls3_23764895ai_0. It looks like this task restarted 4 times (wasting cpu) completed successfully, but failed to upload?? Can you check what happened?
ID: 3151 · Rating: 0 · rate: Rate + / Rate - Report as offensive
zombie67 [MM]
Avatar

Send message
Joined: 16 Feb 19
Posts: 14
Credit: 1,519,846
RAC: 0
Message 3159 - Posted: 26 Feb 2019, 15:13:50 UTC - in response to Message 3151.  

Now, please, zombie67, you returned two error tasks: 23764895ii and padls3_23764895ai_0. It looks like this task restarted 4 times (wasting cpu) completed successfully, but failed to upload?? Can you check what happened?


I wish I could help. But the computer was just sitting there crunching as far as I can tell. I did reboot it once to install updates. Maybe that had something to do with the errors?
Reno, Nevada
Team: SETI.USA
ID: 3159 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Tomáš Brada
Project administrator
Volunteer developer
Avatar

Send message
Joined: 3 Feb 19
Posts: 632
Credit: 445,023
RAC: 12
Message 3188 - Posted: 2 Mar 2019, 17:57:56 UTC

Rule 38 has been exhausted, before I was able to build windows app. I guess I do not have to worry about the windows app now :D
Next are rule 15 and 51. But I do not want to create two more apps for this. Each app also requires new version of assimilator. These edits are minor, but it still needs to be built on three platforms, add new app to project and add the app versions to project. Instead I will bundle both versions of the generators and select based on argument.

But there is more. There is a third generator, that is not yet in the project. Either run it after 15 and 51 is over, or run it together. Two generators are ran in one workunit, because their outputs intersects, so the intersection is only processed once. I suppose the third generator does intersect as well. Another challenge is that the third generator produces over a million of cf dls per row. There are multiple options.

First: Put all three generators, implement check-pointing and have ridiculously long tasks (talking about 40 hours). There is will be at least 200MiB of RAM wasted for just storing the set. Also the chance of updates or user events killing the app are high in this time range.

Second: Do the rules 15 and 51 with the two generators, like we do now, including the splitting to keep task size 20-40 minutes without checkpoint. Then worry about the third generator later. This works quite well, but I fear it will be over quickly.
Then the third generator will get it's own app. All three rules in the app, but not at the same time. The outer loop of the generator will be restricted to just one index to keep the task size comfortable. Also the first and second generator will run, but their results will be subtracted from the set (to avoid computing the intersection twice).

Third: The subtraction is kind of silly, so unify all three of them in one task. Keeping comfortable task size will be more complicated, she may want the results separate and we already did rule 58 so second option is unavoidable here.

Some automation would help, like Jenkins. I mean, I would commit the source and it would auto-build for all the platforms and add the binaries to boinc. Maybe I could use boinc project to build apps for boinc projects... hmmm. I expected the project to be more set-and-forget, like I would load a batch of wus and it would run for two months.

In conclusion I need sources for gen 1 and 2 for rules 15 and 51 (4 total) for the first part. The third gen would be nice to see as well.
ID: 3188 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Natalia Makarova
Project scientist
Avatar

Send message
Joined: 8 Feb 19
Posts: 356
Credit: 0
RAC: 0
Message 3189 - Posted: 2 Mar 2019, 18:58:34 UTC - in response to Message 3188.  
Last modified: 2 Mar 2019, 19:17:44 UTC

In conclusion I need sources for gen 1 and 2 for rules 15 and 51 (4 total) for the first part. The third gen would be nice to see as well.


Why do you need to see all the sources at once:
1. rule15
2. rule51
3. generator 3
???

As I understand it, you can not run simultaneously 2-3 Applications.
You start the batch and everyone is waiting for this batch to end and credits will be assigned.
What happens if you run a batch that will be processed for 2-3 months?
Who wants to wait for loans for 2-3 months???

I understand the experiment completely.
The experiment has many different conditions.
Generator 3 is not everything!
There is a total generator.
Maybe you will start immediately with a total generator? :)
Run WUs and the project will work for 2-3 years!
And all this time, people will wait for loans.
I understand correctly?

Next are rule 15 and 51. But I do not want to create two more apps for this. Each app also requires new version of assimilator. These edits are minor, but it still needs to be built on three platforms, add new app to project and add the app versions to project. Instead I will bundle both versions of the generators and select based on argument.

Sorry, I do not understand.

I expected the project to be more set-and-forget, like I would load a batch of wus and it would run for two months.

Since the project is incomprehensible to you, I suggest you stop it.
I already have enough. I was convinced of the effectiveness of my algorithm. It works great!

If you remember, I have several algorithms besides the PADLS algorithm.
However, each new algorithm requires the creation of a new Application.
It is impossible to make a universal Application for all algorithms!

PS. Thank very much for your trial BOINC project.

I realized that there would be no experimental BOINC project.
There will be only two BOINC projects (ODLK and ODLK1) that work with permanent Applications.
No one wants to create new Applications!
We came to the point where we started.
Great humor :)
ID: 3189 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Tomáš Brada
Project administrator
Volunteer developer
Avatar

Send message
Joined: 3 Feb 19
Posts: 632
Credit: 445,023
RAC: 12
Message 3190 - Posted: 2 Mar 2019, 19:20:49 UTC - in response to Message 3189.  

I was thinking aloud about the next progress of this server. I hope you are not upset.
It is up to you to send me sources and new experiments. And I can load the experiment on this server (to be run by contributors).
ID: 3190 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Natalia Makarova
Project scientist
Avatar

Send message
Joined: 8 Feb 19
Posts: 356
Credit: 0
RAC: 0
Message 3191 - Posted: 2 Mar 2019, 19:25:37 UTC

Tomas Brada
by the way, generators 1 and 2 and the WUs array for PADLS rule 15 are ready for a long time.
I offered to send it to you, but you said that you can create a new Application only next week.

Today I started processing WUs for PADLS rule 15 on my PC.
I think it will have to continue.

I don’t dare to bother you with new Applications.
ID: 3191 · Rating: 0 · rate: Rate + / Rate - Report as offensive
1 · 2 · 3 · Next

Message boards : Code and Servers : New generators

©2021 Tomáš Brada