PADLS batch 10 complete

Message boards : News : PADLS batch 10 complete
Message board moderation

To post messages, you must log in.

AuthorMessage
Tomáš Brada
Project administrator
Volunteer developer
Avatar

Send message
Joined: 3 Feb 19
Posts: 632
Credit: 445,023
RAC: 12
Message 3075 - Posted: 22 Feb 2019, 14:24:06 UTC
Last modified: 4 Mar 2019, 10:54:28 UTC

Last result from first batch of 1000 rows was completed today. The results were collected and sent for processing. After examining the results, new tasks will be created. 12665 CF ODLS were found. Post processing will likely reveal more.

Average number of ODLSs per row was 20.6. But users fzs600, Ralfy, DoctorNow and XAVER processed rows with 145, 237 and 245 odls.

Today I tested a new (first) assimilator for results. This assimilator has, besides from assimilating, new credit awarding algorithm. Credit for completed tasks stays the same, but from today on, credit will not be awarded during validation, but later during assimilation of the batch. Credit will be granted based on the number of Latin squares checked.

You could already have noticed that I added a batch summary in the server status page.

I noticed that too many tasks are being assigned to slower computers. For this reason, I will be decreasing the deadline for next batch. It would be nice if I could tell the server to extend deadline for tasks that are actively running and reschedule tasks that are sitting deep in queues of busy computers.

Results: https://boinc.tbrada.eu/download/padls_10.zip.
Copyright: Natalia Makarova, Tomáš Brada. Attribution to Natalia Makarova is required for any derivative work.
ID: 3075 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Natalia Makarova
Project scientist
Avatar

Send message
Joined: 8 Feb 19
Posts: 356
Credit: 0
RAC: 0
Message 3077 - Posted: 22 Feb 2019, 15:23:04 UTC - in response to Message 3075.  
Last modified: 22 Feb 2019, 15:59:37 UTC

Tomas Brada
please see the mail.
I can not open your file with a strange extension
padls_10.tar.xz

Please send a file with the standard rar or zip or txt extension.
ID: 3077 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Natalia Makarova
Project scientist
Avatar

Send message
Joined: 8 Feb 19
Posts: 356
Credit: 0
RAC: 0
Message 3078 - Posted: 22 Feb 2019, 15:29:52 UTC - in response to Message 3075.  
Last modified: 22 Feb 2019, 15:30:20 UTC

Last result from first batch of 1000 rows was completed today. The results were collected and sent for processing. After examining the results, new tasks will be created. 12665 CF ODLS were found. Post processing will likely reveal more.

I did not understand what the “12665 CF ODLS were found” is.
Are these raw results?

In your project, there will also be no automatic processing of the results ???
You have described the processing software so well!
You have a total database of ODLK and ODLK1 projects.

I can not check the uniqueness of the solutions relative the total database.
ID: 3078 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Natalia Makarova
Project scientist
Avatar

Send message
Joined: 8 Feb 19
Posts: 356
Credit: 0
RAC: 0
Message 3079 - Posted: 22 Feb 2019, 15:49:24 UTC
Last modified: 22 Feb 2019, 15:52:32 UTC

Tomas Brada
please don't forget it

Addition to the database from the autonomous subproject

21203 unique CFs ODLS

November 5, 2018 - February 10, 2019

©2019 (C) Natalia Makarova & participants of autonomous subproject

Link
https://cloud.mail.ru/public/6LMT/Ngt15i43n

Created on February 10, 2019


You do automatic processing of project results now!
We will not delay it.

Progger and ice00 have not done automatic processing yet.
All this time I processed the results of the ODLK and ODLK1 projects manually.
ID: 3079 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Tomáš Brada
Project administrator
Volunteer developer
Avatar

Send message
Joined: 3 Feb 19
Posts: 632
Credit: 445,023
RAC: 12
Message 3081 - Posted: 22 Feb 2019, 16:00:33 UTC

.tar.xz files can be opened using 7Zip, WinRar, WinZip, PeaZip and others.
I loaded addition_results_67879.txt (67879), add_aut_subproject_21203.txt (21203) to new database specially for this experiment.
This batch file padls_10_out.txt, gives:
local dup: 7962
local unique: 12665
global dup: 8841
global new: 3824

Now I started the post-processing program on 3824 cfodlk. Process running:
Read 3824 squares from input
Iteration 0 input set: 3824
cf lk 3474
Progress: step 0  46.0%    2/s * 4 threads
ID: 3081 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Natalia Makarova
Project scientist
Avatar

Send message
Joined: 8 Feb 19
Posts: 356
Credit: 0
RAC: 0
Message 3082 - Posted: 22 Feb 2019, 16:06:59 UTC - in response to Message 3081.  
Last modified: 22 Feb 2019, 16:18:48 UTC

.Now I started the post-processing program on 3824 cfodlk. Process running:
Read 3824 squares from input
Iteration 0 input set: 3824
cf lk 3474
Progress: step 0  46.0%    2/s * 4 threads

You need to do all the processing in automatic mode in the project - from beginning to end, including checking for uniqueness and outputting groups of pairs of ODLS (k>2), as well as the formation of a new version of the database.
ID: 3082 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Natalia Makarova
Project scientist
Avatar

Send message
Joined: 8 Feb 19
Posts: 356
Credit: 0
RAC: 0
Message 3083 - Posted: 22 Feb 2019, 16:10:09 UTC - in response to Message 3081.  

.I loaded addition_results_67879.txt (67879), add_aut_subproject_21203.txt (21203) to new database specially for this experiment.

This is not the whole DB.

Ideally, you should check against the total database. And then create a new version of the database, adding unique solutions.
ID: 3083 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Natalia Makarova
Project scientist
Avatar

Send message
Joined: 8 Feb 19
Posts: 356
Credit: 0
RAC: 0
Message 3084 - Posted: 22 Feb 2019, 16:15:16 UTC - in response to Message 3081.  

.tar.xz files can be opened using 7Zip, WinRar, WinZip, PeaZip and others.

File does not open in any of the programs except Notepad, but this is a meaningless set of characters.

Please send me files with standard rar, zip or txt extensions.
ID: 3084 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Tomáš Brada
Project administrator
Volunteer developer
Avatar

Send message
Joined: 3 Feb 19
Posts: 632
Credit: 445,023
RAC: 12
Message 3085 - Posted: 22 Feb 2019, 16:22:32 UTC - in response to Message 3082.  

You need to do all the processing in automatic mode in the project - from beginning to end, including checking for uniqueness and outputting groups of pairs of ODLS, as well as the formation of a new version of the database.

I agree. Automated processing would be best.

About the database... so there is the global database and database for this project should be separate. But new additions should be checked for uniqueness also against the global database, right? And can the results from this project be included in the global database as well?

Cool, we have the boinc assimilator semi-automated (some error cases are not solved),
then checking against two databases - this can be automated
then the zamyk process, which would be run as boinc app (linux only)
then database checking again
and lastly outputting the pair groups - will be automated.
ID: 3085 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Natalia Makarova
Project scientist
Avatar

Send message
Joined: 8 Feb 19
Posts: 356
Credit: 0
RAC: 0
Message 3086 - Posted: 22 Feb 2019, 16:31:51 UTC - in response to Message 3085.  
Last modified: 22 Feb 2019, 16:32:07 UTC

About the database... so there is the global database and database for this project should be separate. But new additions should be checked for uniqueness also against the global database, right?

Yes.

And can the results from this project be included in the global database as well?

Since the administrators do not perform the processing of results in the ODLK and ODLK1 projects, the database will not be changed there. Right?

So you better make a separate database in your BOINC project.
However, the uniqueness check is required against the existing total database.
ID: 3086 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Natalia Makarova
Project scientist
Avatar

Send message
Joined: 8 Feb 19
Posts: 356
Credit: 0
RAC: 0
Message 3091 - Posted: 22 Feb 2019, 18:57:52 UTC

Tomas Brada
I got the results, the archive open and I analyz.
Later I will report the results.

Thanks.
ID: 3091 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Natalia Makarova
Project scientist
Avatar

Send message
Joined: 8 Feb 19
Posts: 356
Credit: 0
RAC: 0
Message 3094 - Posted: 22 Feb 2019, 19:45:02 UTC
Last modified: 22 Feb 2019, 19:48:01 UTC

Tomas Brada
there is no out_klpmd.txt file in the archive
This is the desired file.

Please send me this file.

PS. I found unprocessed results in the archive. Yes, it is 12,665 CF.
ID: 3094 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Tomáš Brada
Project administrator
Volunteer developer
Avatar

Send message
Joined: 3 Feb 19
Posts: 632
Credit: 445,023
RAC: 12
Message 3096 - Posted: 22 Feb 2019, 20:20:18 UTC - in response to Message 3094.  
Last modified: 22 Feb 2019, 21:25:27 UTC

There is no out_klpmd.txt file in the archive This is the desired file.

out_klpmd.txt is not produced by my post-processing program. If you combine padls_10_pre.txt and padls_10_pp.txt you will get what would be in out_klpmd.
The padls_10_pre.txt are unique cf odls from project and padls_10_pp.txt is basically out_klpmd.txt without squares already in padls_10_pre.txt.
The files padls_10_pre.txt and padls_10_pp.txt are additions to the database from this batch. They are split, because that is how the processing makes them.

Edit: it does not work. I need to work on the processing some more because the files are incomplete.
ID: 3096 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Profile Henk Haneveld

Send message
Joined: 17 Feb 19
Posts: 2
Credit: 40,389
RAC: 37
Message 3097 - Posted: 23 Feb 2019, 7:26:38 UTC - in response to Message 3075.  

I noticed that too many tasks are being assigned to slower computers. For this reason, I will be decreasing the deadline for next batch. It would be nice if I could tell the server to extend deadline for tasks that are actively running and reschedule tasks that are sitting deep in queues of busy computers.


I picked this up from the SRbase project forum. The message was from the admin.

I have added
<grace_period_hours>48</grace_period_hours>
to the server config. This means if a result is over the deadline it has 48h more time to report before a new replication is created and sending out.


The Primegrid also uses something like this, so there must be a server setting present.
ID: 3097 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Tomáš Brada
Project administrator
Volunteer developer
Avatar

Send message
Joined: 3 Feb 19
Posts: 632
Credit: 445,023
RAC: 12
Message 3100 - Posted: 23 Feb 2019, 23:56:49 UTC - in response to Message 3097.  
Last modified: 23 Feb 2019, 23:57:25 UTC

Thanks. I will add a grace period and reduce the delay_bound parameter. There is also reliable_on_priority family of options, which looks promising. Alternatively I can create a database command to increase priority when batch is near end.
ID: 3100 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Natalia Makarova
Project scientist
Avatar

Send message
Joined: 8 Feb 19
Posts: 356
Credit: 0
RAC: 0
Message 3103 - Posted: 24 Feb 2019, 7:08:45 UTC

Tomas Brada
I got the raw results of raw.txt and processed them with the zamyk.bat script.

I will show the result of the processing (but this is before the removal of non-unique solutions!)

Найдено марьяжных КФ:
count[1] = 12151
count[2] = 1126
count[3] = 9
count[4] = 60
count[6] = 5
count[7] = 1
count[8] = 2
Всего: 13354
Найдено соквадратов: 14723
КФ соквадратов: 13354

We need to get unique results.
count[6], count[7] and count[8] are not unique.
It seems there are two unique count[3] and two unique count[4].
But it is required to clarify.

According to the results, we see that the experiment gives excellent solutions. But many of them have already been found before.

It is unfortunate that you do not give me the file out_klpmd.txt.
This is the main result of processing.
To get this file, I need to re-process raw results the zamyk.bat script.

We are waiting for the completion of the processing of the second batch of WUs.
ID: 3103 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Natalia Makarova
Project scientist
Avatar

Send message
Joined: 8 Feb 19
Posts: 356
Credit: 0
RAC: 0
Message 3104 - Posted: 24 Feb 2019, 7:25:03 UTC
Last modified: 24 Feb 2019, 7:31:15 UTC

In the first batch of WUs found two unique groups of three pairs ODLS

0 4 3 5 7 8 2 9 6 1
5 1 9 6 0 3 7 4 2 8
6 7 2 8 1 9 5 3 4 0
2 5 1 3 9 7 4 8 0 6
9 3 8 0 4 6 1 2 7 5
1 9 7 4 8 5 0 6 3 2
3 8 4 9 2 1 6 0 5 7
8 2 5 1 6 0 3 7 9 4
4 0 6 7 5 2 9 1 8 3
7 6 0 2 3 4 8 5 1 9

0 4 3 7 5 8 2 9 6 1
5 1 9 6 0 3 7 4 2 8
6 7 2 8 1 9 5 3 4 0
2 5 1 3 9 7 4 8 0 6
9 3 8 0 4 6 1 2 5 7
1 9 7 4 8 5 0 6 3 2
3 8 4 9 2 1 6 0 7 5
8 2 5 1 6 0 3 7 9 4
4 0 6 5 7 2 9 1 8 3
7 6 0 2 3 4 8 5 1 9

and two unique groups of four pairs ODLS

0 2 5 9 6 3 7 8 4 1
2 1 8 4 5 6 3 9 0 7
4 8 2 5 7 1 0 3 9 6
8 7 0 3 1 9 4 5 6 2
3 9 6 1 4 7 8 2 5 0
1 4 7 6 2 5 9 0 3 8
7 5 1 8 9 0 6 4 2 3
6 3 9 0 8 2 5 7 1 4
9 6 3 7 0 4 2 1 8 5
5 0 4 2 3 8 1 6 7 9

0 4 6 9 5 3 8 2 7 1
6 1 4 0 8 9 3 5 2 7
1 9 2 5 6 8 7 0 3 4
2 5 0 3 1 7 4 8 9 6
7 0 3 2 4 6 1 9 5 8
3 6 8 4 7 5 9 1 0 2
9 7 5 8 2 0 6 4 1 3
8 2 9 6 3 1 0 7 4 5
4 3 1 7 9 2 5 6 8 0
5 8 7 1 0 4 2 3 6 9

These are groups of three pairs ODLS, they are twins.



Very interesting solution!

See
https://boinc.progger.info/odlk/forum_thread.php?id=42
https://boinc.progger.info/odlk/forum_thread.php?id=40
ID: 3104 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Tomáš Brada
Project administrator
Volunteer developer
Avatar

Send message
Joined: 3 Feb 19
Posts: 632
Credit: 445,023
RAC: 12
Message 3105 - Posted: 24 Feb 2019, 9:05:30 UTC - in response to Message 3103.  
Last modified: 24 Feb 2019, 9:09:18 UTC

You can recreate the out_klpmd file by combining out_kf_x files from the archive. Or do you also want the duplicate results?

Yes, There are two unique count[4]:
EVy4xt2jEFwMXHpdVCFdK6Qe1
M95CAkD4SSM3eYSb21k6F41YQ
And two unique cont[3]:
UZ96gRAftFCFyCxVs5NBhGe8q
UZ4EKS6bLFCFxWTJ581vPBhyk
Then 73 of 2 and 4976 of 1.
ID: 3105 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Natalia Makarova
Project scientist
Avatar

Send message
Joined: 8 Feb 19
Posts: 356
Credit: 0
RAC: 0
Message 3106 - Posted: 24 Feb 2019, 9:20:42 UTC - in response to Message 3105.  

Or do you also want the duplicate results?

No, I do not want to duplicate the database of your project.

I think you will make a good DB.
ID: 3106 · Rating: 0 · rate: Rate + / Rate - Report as offensive
Natalia Makarova
Project scientist
Avatar

Send message
Joined: 8 Feb 19
Posts: 356
Credit: 0
RAC: 0
Message 3107 - Posted: 24 Feb 2019, 9:55:41 UTC - in response to Message 3105.  

Yes, There are two unique count[4]:
EVy4xt2jEFwMXHpdVCFdK6Qe1
M95CAkD4SSM3eYSb21k6F41YQ
And two unique cont[3]:
UZ96gRAftFCFyCxVs5NBhGe8q
UZ4EKS6bLFCFxWTJ581vPBhyk
Then 73 of 2 and 4976 of 1.

See this message for coded format
https://boinc.multi-pool.info/latinsquares/forum_thread.php?id=54&postid=697#697

For example

EVy4xt2jEFwMXHpdVCFdK6Qe1
M95CAkD4SSM3eYSb21k6F41YQ -->

0 2 5 9 6 3 7 8 4 1
2 1 8 4 5 6 3 9 0 7
4 8 2 5 7 1 0 3 9 6
8 7 0 3 1 9 4 5 6 2
3 9 6 1 4 7 8 2 5 0
1 4 7 6 2 5 9 0 3 8
7 5 1 8 9 0 6 4 2 3
6 3 9 0 8 2 5 7 1 4
9 6 3 7 0 4 2 1 8 5
5 0 4 2 3 8 1 6 7 9

0 4 6 9 5 3 8 2 7 1
6 1 4 0 8 9 3 5 2 7
1 9 2 5 6 8 7 0 3 4
2 5 0 3 1 7 4 8 9 6
7 0 3 2 4 6 1 9 5 8
3 6 8 4 7 5 9 1 0 2
9 7 5 8 2 0 6 4 1 3
8 2 9 6 3 1 0 7 4 5
4 3 1 7 9 2 5 6 8 0
5 8 7 1 0 4 2 3 6 9
ID: 3107 · Rating: 0 · rate: Rate + / Rate - Report as offensive

Message boards : News : PADLS batch 10 complete

©2021 Tomáš Brada