Testing new LPoS distribution tools

hokusai.ch · September 16, 2018, 12:15pm

I decided to brush up on my JavaScript skills and as a pet project I started working on set of tools that handle a Waves node’s accounting and payments. Currently this is beta quality software and it’s not advisable to use it on Mainnet. The code is being released to gather feedback from the community and get it into a production-ready state. You can find the code in github, the project is called WavesLPoSTools.

This project is modeled on Marc Jansen’s WavesLPoSDistributer and follows the same conventions as much as possible. The objective is being a drop in replacement. Following the same architecture has the added bonus of being able to verify the correctness of each script’s output.

The code was written from scratch, with the intent of improving several areas:

providing easier, centralized configuration that is isolated in a separate file;
lowering memory and disk requirements;
lowering execution time;
allowing data analysis and statistics on top of the blockchain information

There is some overlap with functionality provided by Waves data service API and Waves blockchain — PostgreSQL sync scripts, but these projects currently don’t provide all the information required to calculate a node payout (namely the lease information).

The code is written in Javascript using modern paradigms like asynchronous I/O and functional programming. It uses ES6 Promises and ES7 async/await to provide the best performance.

As in WavesLPoSDistributer you can use the payment utilities to execute payments for any file that conforms to the payout.json format. So if you have any home brew utilities that write their output in this format you can keep on using them as usual.

Any ideas on how to improve, pull requests, or any kind of constructive feedback is accepted. A couple of ideas for future versions are:

Airdrop script
database support for more transaction types
a REST interface to extract information from the database

deemru · September 16, 2018, 2:22pm

Did you try this tool: W8io — Waves explorer based on PHP + SQLite

It works in just a seconds on any leasing pool.

hokusai.ch · September 16, 2018, 3:37pm

I didn’t know that project, thanks! I’ll take a look Anyway I wanted to learn more about modern JavaScript server-side development, thus my choice. I know PHP but it’s popularity is decreasing due to a few known issues.

Maybe in the future I’ll change the code to retrieve the statistics with Waves data service API and Waves blockchain — PostgreSQL sync scripts which are being developed by the Waves Platform team. The payment scripts will still be useful I hope.

Raul_localwavesnode · September 16, 2018, 9:36pm

I will try
Thanks for your job ;D

Raul_localwavesnode · September 17, 2018, 10:59pm

WavesLPoSTools# npm install
npm WARN [email protected] No repository field.

audited 471 packages in 3.836s
found 0 vulnerabilities

hokusai.ch · September 18, 2018, 9:39am

Let me know if you see any issues. Downloading the blockchain and creating the database will take some time, but after that the updates should be quick.

Raul_localwavesnode · September 18, 2018, 3:28pm

Hi, can you put the blocks.db in a URL for downloading?
The database store all nodes entries.
I have synced for more than 14 hours and only sync 316401 blocs :…(

hokusai.ch · September 18, 2018, 4:45pm

Currently I only have a good snapshot of blocks.db for Mainnet (but you shouldn’t use the payments scripts with that one). You can try the db update and the payout calculation though.

The copy of blocks.db I have for Testnet is currently a work in progress, to add more info about other transaction types I have to start from scratch.

Raul_localwavesnode · September 18, 2018, 5:04pm

Thanks, only testing, and compare with LPoSDistributer ;p

Raul_localwavesnode · September 19, 2018, 4:47pm

Hi, I’ve downloaded and use your db and renamed to blocks_full.db
The result is buggy
This is the config file
resultados
I run the script and dump 0 payments

hokusai.ch · September 19, 2018, 5:32pm

As a security precaution remove your apiKey, accidents happen!

I’ve tried calculating the payout for 3P5hx8Lw6nCYgFkQcwHkFZQnwbfF7DfhyyP and payments were dumped. To double check:

You’re calculating the payout for waveslocalnode right?
The file you downloaded is 95 424 512 bytes?
What version of nodejs are you using?

Thanks!

hokusai.ch · September 19, 2018, 6:43pm

I think I’ve spotted the issue @Raul_localwavesnode instead of startBlockHeight use startBlock in the configuration.

I’m going to correct the typo in config.json.sample immediately and push it to github.

I will also need to understand why the script doesn’t complain when a a required option is absent

Thank you so much for the feedback!

Raul_localwavesnode · September 19, 2018, 11:20pm

Ok, thanks to you too for developing open source

hokusai.ch · September 28, 2018, 2:32pm

Hello @Raul_localwavesnode have you confirmed that things are working correctly after the changing that parameter name?

I want to include you in the Acknowledgements sections of the README.md for the project, how should I refer to you? A name and a link would be perfect.

Thanks

Raul_localwavesnode · September 28, 2018, 2:52pm

Yes of course, It works!
Raúl (@d00han_) from localwavesnode.com

hokusai.ch · September 28, 2018, 4:00pm

I’ve linked to your github profile and I mention localwavesnode.

In the next few days I’ll add some more updates to the code. I’m making changes to enable our node’s new policy (distributing 100% of the fees but subtracting the cost of transferring the payout to leasers).

Raul_localwavesnode · October 8, 2018, 9:55am

Please, notify when it be finished. I will test

hokusai.ch · October 8, 2018, 5:21pm

I made a very small change…

Command line options are now used by payout.js to provide values that change frequently:

start block for the calculation
end block for the calculation
the amount of Miner’s Reward Tokens that should be payed by generated block

hokusai.ch · November 28, 2018, 4:15pm

Following some thorough testing and some bug fixing the code is now production ready!

A couple of changes:

Updated the Acknowledgments section of the README to mention everybody that helped with testing
Added option to specify the batch size when retrieving block information

Get it on WavesLPoSTools gitbub page. If you’re having trouble with downloading the blockchain information try adding the batchSize option to your config file.

wallison · November 29, 2018, 11:17pm

Is there any way not to pay Leasint’s rewards from a specific address? Example, XXY address allocated 1000 Waves but is a wallet my stay that is always offline, and I want the reward to be sent directly to another YYX wallet. Would you have to put this function in the program?