Synergy - Project SOLID and SAFEnetwork

[happybeing]

See proposal of related ideas in topic by @DavidMtl.

App idea: Safe Layer. Let's talk about the data

And how these relate to Solid + SAFE:

App idea: Safe Layer. Let's talk about the data

[joshuef]

Hey @happybeing!

I’ve fiiiiiiiiinally been getting to some more serious SOLID reading, and digging into your services lib a bit (nice work so far!). I have a few Qs I was hoping you might be able to clarify:

• The aim for the lib is to hijack fetch and would be the REST API for SOLID, in order to map those to SAFE functionality (MDs)? Or are more things at work?

• You’re using rdf.js to manage the triples and perform SPARQL queries on returned data?

• The advantage of rdf.js is the above exactly? Simpler queries of linked data right? Or is rdf.js more focussed on generating REST style calls?

Let me know if I’m on/near/wayoff there, please

[happybeing]

That’s pretty much it. Glad you are looking at this Josh as I would value any feedback, observations, ideas, advice etc

To confirm/clarify:

• rdflib.js is forked and modified very slightly so I can intercept its fetch()
• a Solid app expects to use HTTP to talk to a RESTful (Solid) server
• a Solid app can use fetch(), or the rdflib.js Fetcher (which adds functionality)
• if it uses other means (e.g XHmtlRequest) they must first be converted to fetch()
• the Solid/SAFE Plume demo uses both window.fetch() and rdflib.js Fetcher
• safenetwork-webapi:
  • implements LDP (a subset of Solid server)
  • lacks WebID auth or access control lists
  • intercepts any window.fetch() or Fetcher and handles any safe: URLs
  • is written with more than Solid/LDP in mind. My idea is that additional services could be supported by just adding a ServiceInterface class (file sharing, remoteStorage.io etc)

rdflib.js is extensive and I don’t know it very well. The Fetcher handles different data formats, and returns a ‘graph’ which can then be queried directly. Also, I think the Fetcher may be able to crawl multiple resources where one resource refers to another, including on other domains - a bit like a scraper where you set the depth. I’m not sure about that, so take with a pinch of salt, but if so how cool is that!

The SAFE Plume demo uses Fetcher, but doesn’t do anything complex, so I think it could just as well only use fetch() but I may be underestimating it!

[joshuef]

@happybeing

I think the Fetcher may be able to crawl multiple resources where one resource refers to another, including on other domains

Yeh nice. Sounds similar to how I was imagining search indexes to function (time and time ago when I was trying to whip that up back when Structured Data was a thing ).

---

@happybeing, it’s a great idea for smoothing out things for SOLID devs to open up adoption etc

Only thing that pops up is that right now you’re working in one JS file. For end use probably it should be split up, such that you need only include the LDP portion and wont need the rest of the code services etc that are irrelevant for you.

---

Mostly for me it leads into questions about how we should be storing the data on SAFE, and if/how we can treat LDP as something first class perhaps… ? So in the end we could aim to be building apps directly on the network using this setup for data.

How are you storing/writing data on the network at the moment?
An array of objects on the MD?

{
    //... rest of MD stuff
    RDF : [ //... more triples
        {
            s : 'SAFE'
            p : 'is'
            o : 'rad'
        },
        {
            s : 'happybeing'
            p : 'made'
            o : 'this'
        }
    ]
}

Is there any more info that needs to be stored for Linked Data?

---

Some other thoughts:

• I wonder how LD triples lend themselves to site indexing (although probably there’d be a need for more detailed info than RDF triples provides?)

• What would be the expected / potential structures of LD for a website or a user profile? (is that standardised anywhere?)

• Could we query the network directly for triples? Is that even possible? Is that even desirable?

[happybeing]

Thanks for the feedback and support Josh, I’m glad you’re interested in this and it would be great to have as much input from you as you want to / can give

You ask important questions and ideas which are food for thought. Some answers & comment…

The library is one file just for convenience atm but the code is structured so it can be split up easily, new services added, implementable replaced etc, even by non coders. So yes, that’s to be done.

I’ve also just fleshed my thoughts on how to standardise this for SAFE, by using RESTful file based services as you may have seen:

Proposals for RESTful service handling

SAFE Network Services Currently SAFE services are created by inserting an entry into the services MD of a public name. For example, for the public name ‘thing’ the entries in the corresponding services MD might be: Key Explanation blog@www There’s a website at safe://blog.thing www There’s a different website website at safe://www.thing which you can also reach at safe://thing mail@email You can send email to an inbox at mail.thing I propose a modification to this convention for…

The LDP and Solid standards use a file / document metaphor, which are resources in RDF terminology. So this is how the storage is implemented, and the transactions boil down to CRUD operations on containers and files, but using RDF (usually Turtle for any payload and response body). Turtle is essentially a text format for Subject/Predicate/Object triples (e.g. Josh/works for/Maidsafe etc - BTW that’s not Turtle, just a hint about what a triple is ).

Linked Data is generally implemented in a Solid or LDP server as directories of files where each file represents an RDF resource that can be addressed with its own URI. Responses are therfore often parsed into a graph which is then queried and manipulated.

SPARQL queries return the triples which might be a subset of one resource or drawn from multiple resources, even spread across different servers.

So my implementation stores each RDF resource as a file (using SAFE NFS). Content negotiation (not yet implemented) allows the requester to receive results in different forms, depending on the representation that is stored (Turtle, N3, JSON-LD etc) and whatever conversions the server supports from each stored representation. In addition, rdflib.js Fetcher has several conversions built in too - so client side.

Note that a resource can be in any format, not just RDF, so plain text, .png, .mp3 etc, but using RDF leverages the semantic Web, SPARQL etc and if you upload a Word document to Tim’s Solid server, a kitten dies.

I’ll be showing the RESTful LDP interface at work, and the Turtle it returns for container listings, and that is used store blog posts in my demo. So if you want to see some really simple LD maybe take a look at that - it’s all there in the console output if you visit plumetest atm).

@joshuef

I wonder how LD triples lend themselves to site indexing (although probably there’d be a need for more detailed info than RDF triples provides?)

I don’t think there’s any limit to what triples can represent, though what is the best implementation is a different matter. Some use file system back ends, some graph databases etc.

For leveraging the existing semantic Web (the data out there, applications, tools, libraries and skills), and to create an ecosystem where SAFE apps can share, reuse and reach across a sea of data, it’s the protocols Solid, LDP and the standards RDF/LinkedData, WebID etc that we should support. And things like the RESTful principles and the [Architecture of the World Wide Web] (Architecture of the World Wide Web, Volume One). Sitting in on the chats between Tim and his colleagues has taught me a lot about these things, and the value of building with them.

@joshuef

What would be the expected / potential structures of LD for a website or a user profile? (is that standardised anywhere?)

HTML has various ways of integrating with LinkedData and there is a standard way for users to publish a profile as part of a Web identity (a WebID URI), which is something a user owns and controls.

@joshuef

Could we query the network directly for triples? Is that even possible? Is that even desirable?

Now you’re thinking! Lots of ways to pay with this and I haven’t really got the basics yet. Step by step I plod. I look forward to working with folks to figure things like this out.

[happybeing]

Latest: video demo of a Solid app running on the SAFE Network and slides from a presentation explaining the benefits of building Solid apps on SAFE.

I spent a few months getting this far, but can’t take this much further on my own so if you like these ideas please consider getting involved - I have lots of ideas for things to do!

DevCon talk Supercharging the SAFE Network with Project Solid

[hunterlester]

How do we move this forward?

Although far from understanding the concepts of the semantic web, the most obvious piece is that of resources being linked and accessed via URI which brings us back again to @bochaco’s to be able to access data on the network as a SAFE URI via XOR name, like safe://4291165faf85f9964d4c8f5d12e4b0dc31ffda4509d2049704c77a018f922c98 (a hexadecimal slice of a 32 byte XOR name).

Additionally, it’s necessary for a URI to contain the format and metadata of the data which it represents, which https://multiformats.io can help provide inspiration.

[happybeing]

I think it will be useful both to have a mechanism such as the one you refer to here, and the ability for RDF resources to be accessed by the browser as if they are a Web service. The former will allow linking directly to Immutable Data, and more general forms of file sharing without using a public ID, so will be very valuable. The latter may prove useful in itself, because a short readable URI is a better UI, and can be updated for example, and is available now.

In the DevCon demo you can see me doing the second kind, but I hacked it by making an LDP based on top a SAFE www device, rather than a separate new LDP SAFE service type - which is necessary if we want to generalise this. I made a proposal for how to do that here:

Proposals for RESTful service handling

SAFE Network Services Currently SAFE services are created by inserting an entry into the services MD of a public name. For example, for the public name ‘thing’ the entries in the corresponding services MD might be: Key Explanation blog@www There’s a website at safe://blog.thing www There’s a different website website at safe://www.thing which you can also reach at safe://thing mail@email You can send email to an inbox at mail.thing I propose a modification to this convention for…

Where files are stored using NFS we can use the file extension to imply the ContentType (which is how a typical Solid server works), but if the SAFE NFS API is extended to allow this to be set as metadata, we could use that instead.

I’m not sure how important that is for ContentType because using the file extension works well enough, but I think it’s still going to be useful to allow NFS file metadata to be set, and I believe this is already planned.

At the same time I’d like to add the option to have MD entries for containers so they can also have metadata, and can also be explicitly created and deleted, because this improves compatibility with LDP and therefore Solid.

Thanks for your input and please keep digging in, it’s great to see. I am due to chat with @Viv and @Krishna about going forward so tagging them with your interest.

[dirvine]

This is an interesting site for looking at LD sets http://lod-cloud.net/

[happybeing]

Linked Data Templates / LinkedDataHub

A couple of links here relevant to the Semantic Web:

https://gitter.im/linkeddata/chat?at=5b10556d4eaffb692d6fadeb

[happybeing]

SPARQL queries in the client! One to bookmark

Comunica

Comunica is a highly modular and flexible meta query engine for the web

GitHub - comunica/comunica: 📬 A knowledge graph querying framework for...

Overview and details:

Comunica

Direct link to Executing a SPARQL Query.

[Nigel]

@happybeing

At the same time I’d like to add the option to have MD entries for containers so they can also have metadata, and can also be explicitly created and deleted, because this improves compatibility with LDP and therefore Solid.

I second this! This would be huge for media!

[happybeing]

A very interesting development from Ruben Verborgh. SAFE could support this or something inspired by it.

Nice additions to the JavaScript toolkit:

LDflex - a domain-specific language for JavaScript that lets you traverse Linked Data graphs like dynamic objects, and

query-ldflex - access Solid data pods using the LDflex language

Oh, and from Callidon:

sparql-engine - a framework for building SPARQL query engines in Javascript: “Build a SPARQL query engine on top of any data storage system”

[happybeing]

Tim Berners-Lee announces sabbatical to found Inrupt, a company to promote the Solid platform and realise his original vision for an open web of data (not just documents.

Let’s be part of this!

One Small Step for The Web - Tim Berners-Lee

[happybeing]

In search of interoperability, here’s a new resource:

Jackson C Morgan (@otherJackson) on X