SAFE Drive - help with testing

JPL · November 12, 2018, 3:53pm

I still get the same error with that command. Tried again, first setting NODE_ENV=testing - same thing.

happybeing · November 12, 2018, 3:54pm

Don’t set NODE_ENV at all (for live). Check it isn’t set in your environment:

env | grep NODE_ENV

Then use:

SAFENETWORKJS_TESTS=testing DEBUG=safe-fuse:ops*,safe-fuse:stubs*,safe-vfs:*,safenetworkjs* node --inspect bin.js

And you should be ok. Fingers crossed

JPL · November 12, 2018, 4:01pm

And you should be ok. Fingers crossed

Fraid not - toes??

Still getting exactly the same thing.

env | grep NODE_ENV

has no output.

happybeing · November 12, 2018, 4:30pm

Um, it often helps to wait a day and then it all just works

You’re getting advanced developer training here you see

I’d like to study the debug output in full, so can you use the following command to capture it and upload the test.log for me.

Male sure you are using Peruse non-dev build and don’t have NODE_ENV set there either. Being paranoid here, it feels like it helps even if it doesn’t

Capture the output with:

SAFENETWORKJS_TESTS=testing DEBUG=safe-fuse:ops*,safe-fuse:stubs*,safe-vfs:*,safenetworkjs* node --inspect bin.js 2>&1 | tee test.log

Thanks for persevering. I’ll have a go with live myself shortly.

JPL · November 12, 2018, 4:41pm

Indeed - I’d be paying a fortune for this ordinarily!

Here’s the file:

happybeing · November 12, 2018, 8:18pm

Hi John, I’m looking at your test.log and it is looking good. At least not easy to find any errors without knowing what point they happened. Can you tell me what the error is you were seeing, presumably from the test script?

JPL · November 12, 2018, 8:37pm

Yes, it’s this one. At least I presume it’s an error as it stops the test before all the website stuff.

happybeing · November 12, 2018, 8:40pm

Thanks, that’s what I need to know. This is a different error though to the earlier ones I think? Or just tired maybe

You are correct that its an error unless you get the “ALL TESTS PASSED” message!

I’m debugging the code I’ve broken today with my bugfixing, so won’t have a look at the log immediately.

JPL · November 12, 2018, 8:58pm

Yeah, different from the original one, but that’s the error I’ve been getting for a while now. It seems able to create one directory but then not a second one, which is strange. Doesn’t happen with Mock.

happybeing · November 18, 2018, 6:24pm

SAFE Drive news: looking good!

https://twitter.com/safepress/status/1064222202705399808

Note tested on live network yet, but its time too cook a celebratory Sunday dinner!

JPL · November 18, 2018, 7:18pm

Test script works for me on live network on Debian 9 - fantastic work!

happybeing · November 18, 2018, 7:46pm

Woah! This is awesome, I didn’t dare try it. Thanks John, OMG this is brilliant. I can’t believe it, it feels too good to be true.

Next is to retest the apps I know didn’t work before: when editing files they’ve already saved to SAFE Drive both LibreOffice Writer and gedit (Gnome editor) were failing to save changes. `vim’ was OK though. Not tonight though, tonight is for celebrating

We can try publishing a git repo now too. I’m busy tomorrow, putting boat into dry dock in the morning, and don’t know if I’ll be able to work on board yet. Should be, but we’ll see.

I’m really chuffed John, thanks for. checking it out.

JPL · November 18, 2018, 7:55pm

I’m also busy tomorrow but might be able to find a couple of hours in the evening. If so I’ll try those things. Let me know if there is anything else needs testing out.

happybeing · November 18, 2018, 8:07pm

Thanks John, if you can do this that would be great and by all means just play until you break it. Different apps will do things in different ways so just trying things and making some notes about what worked or not would help.

I’ll publish git repo and we’ll see if you and others can clone it, make changes, and push them to your own repo. If that works, we have the basis for a decentralised github

Sascha · November 18, 2018, 11:23pm

Now this I too understand.

But thanks a bunch for your hard work ashore, sailor! I do appreciate it!

happybeing · November 19, 2018, 8:34am

Thanks, appreciation helps too. I have been working non stop on this, but aboard not ashore!

JPL · November 19, 2018, 10:23pm

I can create and save pretty much any sort of file into ~/SAFE/_public/tests/data1 which I think was created by the test script. So png (Gimp), text files (vi nano mousepad) ods, odf, odp files (Libreoffice) all work. However If I create a new directory ~/SAFE/_public/tests/data2 I get a ‘Remote I/O Error’ whatever I try to save or copy there. Same with other pre-existing and new folders from another domain. I can’t save files into those either (although not all give the I/O error).

It seems only data1 can be written to at the moment.

Hope this helps.

happybeing · November 20, 2018, 10:09am

Yes it helps!

Saving to ~/SAFE/_public/tests/data2 won’t work because there is no container there, and I don’t create one automatically. I’m still not sure that’s a good idea, but can’t think of an alternative because the UI is so limited.

So anything under ~/SAFE/_public/tests/data1 should work.

Also anything under an existing path where you already have uploaded a file. So if you have ~/SAFE/_public/anypublicname/www-root you should be able to save under that, but not at ~/SAFE/_public/anypublicname because the container is actually at ~/SAFE/_public/anypublicname/www-root and the directories above it don’t really exist. This is confusing and there’s no way for me to indicate this or explain this to the user, which is why it’s tricky to figure out how best to do it.

From what you said it sounds like you can’t create a file under ~/SAFE/_public/anypublicname/www-root. Is that correct, or were you trying a directory above that level?

Testing different apps and file types is great, thanks. Different sized files, different ways of doing things all help.

One thing I found was not working with LibreOffice Writer and gedit was this:

start app, create file, save to SAFE Drive
close the app and restart it
load the file you saved on SAFE Drive, modify it and save the changes
close and restart the app, reload the file and check the changes were saved.

Basically it helps if anyone is willing, to try as many different apps, or ways of copying, moving, saving, editing etc as possible. And check it has actually done what you expect, even if there is no sign of an error.

This is a lot of tedious work, so I’m not expecting anyone, especially you John, to do this, but it does help a lot.

Files are precious, and we need SAFE Drive to be rock solid or it is no use at all.

Once again John, thanks a lot for your help. I really appreciate it.

I’ve been getting tired with working on this non-stop so I took a break yesterday. It’s the first time it felt OK to step back from the code, feeling that I’ve got a good solution, and understand how it works now (it is complex in places), and how to track down bugs fairly efficiently.

You know that scene from The Matrix where suddenly Neo can read the code. Well I’m now like that with the console output from all my debug statements!

I could do with a bigger break, but there’s a lot still to do, and I know there will be bugs to discover and fix, so I will be pressing on, today or before long.

isntism · November 20, 2018, 2:41pm

I would gladly attempt to help test the SAFEdrive, I have both a windows and (buggy) ubuntu installation, a bad connection, both a laptop and desktop and of course a good amount of persistence.

I do however lack most of the knowledge required, so I’ll attempt to compile and install it using the info on the github, in the meantime if anyone has any pointers for someone with minimal command skills, please do share them.

edit: Ok after an hour or so of fidgeting with gibberish I seem to have gotten it working, compiled from git and now see the _public and _publicNames folder in the drive, also noticed that it used a handfull of puts, but I can’t find any files in the folders, I assumed that this would show you all files, not just ones from the app, was I mistaken or did I make a mistake?

happybeing · November 20, 2018, 3:54pm

Great stuff @isntism, persistence is invaluable with software. The following instructions are all for Linux, and have been tried on Ubuntu and Debian, and to a lesser extent on other distros. If you can help with Windows and MacOS, see below.

For you and anyone who fancies a go with Linux, I recommend:

follow the instructions on the github repo to get the basics set up (installing dependencies, using npm link with SafenetworkJs etc), but to start at least, don’t worry about doing a build. If you don’t get on with that, explain where you got to, what you’ve tried, and ask for help!
get the development branch in safenetwork-fuse and safenetworkjs by doing in each of those directories:

git pull
git checkout development
git checkout .

That will ensure you have the latest code.

Then:

start Peruse browser
login to SAFE Network
in a new terminal console in safenetwork-fuse run:
SAFENETWORKJS_TESTS=testing DEBUG=safenetworkjs*,safe-fuse* node bin.js

(Note: that is for the live network. If you want to save PUTs and use mock, you should run the Dev build of the browser, and include NODE_ENV=dev in the above command.)

At this point you should see some gobbledegook printed to the terminal console, and the browser should jump to the front, requesting authorisation. After you’ve given that, in a new terminal, try:

ls ~/SAFE

And if you see this, give yourself a pat on the back:

_public _publicNames

Success!

If you get to this point, please report back here and mention your setup and anything you think could be done to improve the instructions on github or here. And then play, and again please report back, or post questions and so on.

Bugs n Stuff

If you think you’ve found a bug you can post about it here and/or create an issue on github giving details of your setup, what you did, and what did or didn’t happen. As much detail as possible helps a lot.

For anyone who wants to try, its easy to capture debug output, just add this to the end of the command: 2>&1 | tee test.log and output will be saved to test.log which you can upload or post extracts from.

Windows & MacOS

The above instructions are all for Linux, but we have already made some progress with Windows thanks to @lukas and @benno, and with MacOS thanks to @Stout77. If anyone wants to continue those efforts which are tantalisingly close to working, please post on the following topics:

Debugging

This is complicated code so I don’t expect anyone to take it on, but it can be interesting to some and I do welcome anyone who wants to help with it. Or maybe you just fancy trying the debugger. This is fun (ok, I find it fun) and is well worth trying out if you are interested in how code works or why it does something in a certain way. To use the debugger:

install Chromium (or Chrome) Browser
start Chromium (or Chrome)
get rid of the rude login dialog (I never use it)
paste the following into the address bar: chrome://inspect/#devices
click Open dedicated DevTools for Node
start SAFE Drive as above, but changing node bin.js to node --inspect-brk bin.js

When you start SAFE Drive now, the debugger should open and you will need to press F8 to start the program. You can avoid that step by using --inspect instead of inspect-brk but breaking at the start gives you a chance to set a breakpoint before running the program. If you find the debugger confusing, post a request on this topic.

Oops It Stopped Working

After something fails things may stop working and you will keep getting errors. If that happens try this before running the SAFE Drive command again: pkill node; sudo umount ~/SAFE

Thanks anyone for helping. Everyone can join in if you are willing to have a go. I’m happy to help you whatever level you feel you are at and see if we can get you going. Others are welcome to join in and help people get going of course.

Notes:

SAFENETWORKJS_TESTS=testing causes SAFE Drive to automatically create an NFS container (~/SAFE/_public/tests/data1). This is used by the test script which you can run yourself, although be aware that it uses up to about 400 PUTs per run! So you’ll only be able to run it a maximum of twice on any account. You can run it on mock of course, but will still have to keep creating new accounts while using it (or keep a copy of the MockVault file for an empty account and keep copying that back to /tmp, which is what I do).

DEBUG=safenetworkjs*,safe-fuse* produces output to the terminal that will show what its doing, where it hangs etc. and can be modified to extend and filter what is printed according to the require('debug') statement in different source files. I tend to debug more with the console output than the debugger at the moment because there’s so much useful info printed to show what’s going on.