This HTML5 document contains 67 embedded RDF statements represented using HTML+Microdata notation.

The embedded RDF content will be recognized by any processor of HTML5 Microdata.

Subject Item

n2:OdsBriefcaseAmazonS3

rdf:type

sioct:Comment atom:Entry

dcterms:created

2017-06-13T06:01:46.107419

dcterms:modified

2018-01-02T10:53:36.941512 2017-06-29T07:30:56.780995

rdfs:label

OdsBriefcaseAmazonS3

rdfs:seeAlso

n38:1

foaf:maker

n16:this n27:this

dc:title

OdsBriefcaseAmazonS3

opl:isDescribedUsing

n36:rdf

sioc:has_creator

n31:this n34:this

sioc:attachment

n4:png n6:png n7:png n8:png n10:png n11:png n17:png n20:png n21:png n26:png n28:png n29:png n30:png n32:png n33:png n46:png n47:png

sioc:content

%META:TOPICPARENT{name="OdsBriefcase"}% ---+ Using ODS-Briefcase to mediate access to data stored on Amazon Simple Storage Service (Amazon S3) %TOC% ---++ Introduction Amazon S3 provides a simple web services interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the web. It gives any developer access to the same highly scalable, reliable, secure, fast, inexpensive infrastructure that Amazon uses to run its own global network of web sites. The service aims to maximize benefits of scale and to pass those benefits on to developers. S3 file storage offers users a number of features: * Write, read, and delete objects containing from 1 byte to 5 terabytes of data each. The number of objects you can store is unlimited. * Each object is stored in a bucket and retrieved via a unique, developer-assigned key. * Authentication mechanisms are provided to ensure that data is kept secure from unauthorized access. Objects can be made private or public, and rights can be granted to specific users. * Options for secure data upload/download and encryption of data at rest are provided for additional data protection. * Uses standards-based REST and SOAP interfaces designed to work with any Internet-development toolkit. ODS Briefcase offers a new "Amazon S3" folder type which provides several additional features, once the ODS system is configured with a registered Amazon API key and secret pair: * Specify credentials for authentication against S3, enabling retrieval of any or all of your folders and files, including any associated metadata * Specify which Sponger Extractor Cartridges and Sponger Meta Cartridges should be run against your S3 data * Save fetched data, metadata, and Sponger output, into a newly created S3-specific RDF Graph in the Virtuoso RDF Quad Store * Specify other ODS Users or <nop>WebID-based ACLs which should be granted access to your S3 data through ODS Briefcase ---++ Example Here is a sample scenario showing the whole process, from getting the Amazon S3 Access Key ID and Secret Key, to retrieving all S3 content to ODS-Briefcase. ---+++ Prerequisites * An [[https://aws-portal.amazon.com/gp/aws/developer/registration/index.html/103-7399647-0537426][Amazon Web Services]] (AWS) account, which has been [[http://docs.amazonwebservices.com/AmazonS3/latest/gsg/SigningUpforS3.html][signed up for Amazon S3]] * [[http://docs.amazonwebservices.com/AmazonS3/latest/gsg/GetStartedWithS3.html][Files and folders (data) stored in one of your S3 Buckets]] ---+++ Steps ---++++ 1 - Get an Amazon S3 Access Key ID and Secret Key for use with ODS-Briefcase 1 Browse to the [[https://portal.aws.amazon.com/gp/aws/securityCredentials][AWS "Security Credentials" page]], and log in with your existing AWS user credentials: %BR%%BR%<a href="%ATTACHURLPATH%/step1-01.png" target="_blank"><img src="%ATTACHURLPATH%/step1-01.png" width="600px" /></a>%BR%%BR% 1 Click <b>Access Credentials</b>: %BR%%BR%<a href="%ATTACHURLPATH%/step1-02.png" target="_blank"><img src="%ATTACHURLPATH%/step1-02.png" width="600px" /></a>%BR%%BR% 1 Click <b>Create a new Access Key</b> to get a new Key & Secret pair for use with ODS-Briefcase: %BR%%BR%<a href="%ATTACHURLPATH%/step1-03.png" target="_blank"><img src="%ATTACHURLPATH%/step1-03.png" width="600px" /></a>%BR%%BR% 1 Click <b>Yes</b> when prompted: %BR%%BR%<a href="%ATTACHURLPATH%/step1-04.png" target="_blank"><img src="%ATTACHURLPATH%/step1-04.png" width="600px" /></a>%BR%%BR% 1 Your new Access Key ID will now be displayed. Click <b>Show</b> to see the corresponding Secret Access Key. %BR%%BR%<a href="%ATTACHURLPATH%/step1-05.png" target="_blank"><img src="%ATTACHURLPATH%/step1-05.png" width="600px" /></a>%BR%%BR% 1 You will need both of these for the steps that follow. %BR%%BR%<a href="%ATTACHURLPATH%/step1-06.png" target="_blank"><img src="%ATTACHURLPATH%/step1-06.png" width="600px" /></a>%BR%%BR% ---++++ 2 - Access files stored on Amazon S3 through ODS-Briefcase 1 Browse to your Virtuoso + ODS instance, <code>http://&lt;cname&gt;[:&lt;port&gt;]/ods</code>, e.g., &lt;[[http://example.com/ods/][http://example.com/ods/]]&gt;. %BR%%BR%<a href="%ATTACHURLPATH%/step2-01.png" target="_blank"><img src="%ATTACHURLPATH%/step2-01.png" width="600px" /></a>%BR%%BR% 1 Log in as a Virtuoso+ODS user with normal privileges, e.g., <code>demo</code>. %BR%%BR%<a href="%ATTACHURLPATH%/step2-02.png" target="_blank"><img src="%ATTACHURLPATH%/step2-02.png" width="600px" /></a>%BR%%BR% 1 Click to the <b>ODS-Briefcase</b> application: %BR%%BR%<a href="%ATTACHURLPATH%/step2-03.png" target="_blank"><img src="%ATTACHURLPATH%/step2-03.png" width="600px" /></a>%BR%%BR% 1 Click <b>New Folder</b> from the main Briefcase control strip: %BR%%BR%<a href="%ATTACHURLPATH%/step2-04.png" target="_blank"><img src="%ATTACHURLPATH%/step2-04.png" width="600px" /></a>%BR%%BR% 1 In the New Folder form, specify a <b>Folder Name</b>, e.g., <b><nop>MyAmazonS3Folder</b>, and set the <b>Folder type</b> to <b>Amazon S3</b>: %BR%%BR%<a href="%ATTACHURLPATH%/step2-05.png" target="_blank"><img src="%ATTACHURLPATH%/step2-05.png" width="600px" /></a>%BR%%BR% 1 Click to the newly visible <b>S3 Properties</b> tab: %BR%%BR%<a href="%ATTACHURLPATH%/step2-06.png" target="_blank"><img src="%ATTACHURLPATH%/step2-06.png" width="600px" /></a>%BR%%BR% 1 Set <b>Bucket Name</b> to a specific S3 Bucket Name to restrict ODS-Briefcase access to only this Bucket, or you can leave it empty to give ODS-Briefcase access to all your Buckets. Also enter the Access Key ID and Secret Key values you obtained earlier: %BR%%BR%<a href="%ATTACHURLPATH%/step2-07.png" target="_blank"><img src="%ATTACHURLPATH%/step2-07.png" width="600px" /></a>%BR%%BR% 1 All RDF data extracted from and otherwise collected about your S3 files will be saved to a named graph, where the default name is constructed from the host ODS instance, the ODS username, and the Briefcase folder name -- <code>http://&lt;host&gt;[:&lt;port&gt;]/DAV/home/&lt;user-name&gt;/&lt;folder-name&gt;#this</code> -- so, in this example, the default graph would be named &lt;[[http://example.com/ods/demo/MyAmazonS3Folder#this][http://example.com/ods/demo/MyAmazonS3Folder#this]]&gt;. Expert users may wish to change this graph name, but the default is suitable for most users. %BR%%BR%<a href="%ATTACHURLPATH%/step2-08.png" target="_blank"><img src="%ATTACHURLPATH%/step2-08.png" width="600px" /></a>%BR%%BR% 1 You can also turn on the [[http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSponger][Virtuoso Sponger]], to process your S3 files more comprehensively. When the Sponger is enabled for your folder, you can specify which "Sponger Extractor Cartridges" and "Sponger Meta Cartridges" will be run over the files in S3. You can enable all cartridges, but this will introduce some processing overhead. It's generally best to enable only those cartridges which will be applicable to your files and their content. For instance, you may know that you'll only be storing Microsoft and Open Office documents here. Click <b>Create</b> to finish the folder creation process. %BR%%BR%<a href="%ATTACHURLPATH%/step2-09.png" target="_blank"><img src="%ATTACHURLPATH%/step2-09.png" width="600px" /></a>%BR%%BR% 1 Your new S3-linked folder should now be displayed in your Briefcase resources list: %BR%%BR%<a href="%ATTACHURLPATH%/step2-10.png" target="_blank"><img src="%ATTACHURLPATH%/step2-10.png" width="600px" /></a>%BR%%BR% 1 Click the folder name to see the resources ODS-Briefcase has found in your Amazon S3 Dataspace: %BR%%BR%<a href="%ATTACHURLPATH%/step2-11.png" target="_blank"><img src="%ATTACHURLPATH%/step2-11.png" width="600px" /></a>%BR%%BR% ---+++References * [[http://virtuoso.openlinksw.com/wiki/main/Main/VirtuosoDAVS3][Virtuoso WebDAV S3 support]] * [[http://aws.amazon.com/s3/][Amazon S3 Services]] * [[http://docs.amazonwebservices.com/AmazonCloudFront/latest/GettingStartedGuide/][Amazon Cloud Front Getting Started Guide]] * [[http://bit.ly/SK7WAI][How Linked Data & WebID Deliver Controlled Access to Resources in Google Drive, SkyDrive, DropBox, Box.Net, and Amazon S3]] * [[OdsBriefcaseBoxNet][Using ODS-Briefcase to mediate access to data stored on Box (a/k/a box.com or box.net)]] * [[OdsBriefcaseDropBox][Using ODS-Briefcase to mediate access to data stored on DropBox]] * [[OdsBriefcaseGoogleDrive][Using ODS-Briefcase to mediate access to data stored on Google Drive]] * [[OdsBriefcaseSkyDrive][Using ODS-Briefcase to mediate access to data stored on SkyDrive]] * [[OdsBriefcaseFolders][Manage different Briefcase type folders in Briefcase]] * [[OdsBriefcase][ODS-Briefcase]] * [[http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSponger][Virtuoso Sponger]] * [[http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSources][OpenLink-supplied Virtuoso Sponger Cartridges]] * [[http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMeta][Supported Meta Cartridges in Virtuoso]] * [[http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaREST][Meta Cartridges Usage via REST Request]] * [[http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamples][Parameterized Examples of Meta Cartridge Usage via REST Request]] CategoryVirtuoso CategoryDAV CategoryTutorial CategoryDocumentation CategoryBriefcase CategoryODS CategoryOdsBriefcase %META:TOPICPARENT{name="OdsBriefcase"}% ---+ Using ODS-Briefcase to mediate access to data stored on Amazon Simple Storage Service (Amazon S3) %TOC% ---++ Introduction Amazon S3 provides a simple web services interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the web. It gives any developer access to the same highly scalable, reliable, secure, fast, inexpensive infrastructure that Amazon uses to run its own global network of web sites. The service aims to maximize benefits of scale and to pass those benefits on to developers. S3 file storage offers users a number of features: * Write, read, and delete objects containing from 1 byte to 5 terabytes of data each. The number of objects you can store is unlimited. * Each object is stored in a bucket and retrieved via a unique, developer-assigned key. * Authentication mechanisms are provided to ensure that data is kept secure from unauthorized access. Objects can be made private or public, and rights can be granted to specific users. * Options for secure data upload/download and encryption of data at rest are provided for additional data protection. * Uses standards-based REST and SOAP interfaces designed to work with any Internet-development toolkit. ODS Briefcase offers a new "Amazon S3" folder type which provides several additional features, once the ODS system is configured with a registered Amazon API key and secret pair: * Specify credentials for authentication against S3, enabling retrieval of any or all of your folders and files, including any associated metadata * Specify which Sponger Extractor Cartridges and Sponger Meta Cartridges should be run against your S3 data * Save fetched data, metadata, and Sponger output, into a newly created S3-specific RDF Graph in the Virtuoso RDF Quad Store * Specify other ODS Users or <nop>WebID-based ACLs which should be granted access to your S3 data through ODS Briefcase ---++ Example Here is a sample scenario showing the whole process, from getting the Amazon S3 Access Key ID and Secret Key, to retrieving all S3 content to ODS-Briefcase. ---+++ Prerequisites * An [[https://aws-portal.amazon.com/gp/aws/developer/registration/index.html/103-7399647-0537426][Amazon Web Services]] (AWS) account, which has been [[http://docs.amazonwebservices.com/AmazonS3/latest/gsg/SigningUpforS3.html][signed up for Amazon S3]] * [[http://docs.amazonwebservices.com/AmazonS3/latest/gsg/GetStartedWithS3.html][Files and folders (data) stored in one of your S3 Buckets]] ---+++ Steps ---++++ 1 - Get an Amazon S3 Access Key ID and Secret Key for use with ODS-Briefcase 1 Browse to the [[https://portal.aws.amazon.com/gp/aws/securityCredentials][AWS "Security Credentials" page]], and log in with your existing AWS user credentials: %BR%%BR%<a href="%ATTACHURLPATH%/step1-01.png" target="_blank"><img src="%ATTACHURLPATH%/step1-01.png" width="600px" /></a>%BR%%BR% 1 Click <b>Access Credentials</b>: %BR%%BR%<a href="%ATTACHURLPATH%/step1-02.png" target="_blank"><img src="%ATTACHURLPATH%/step1-02.png" width="600px" /></a>%BR%%BR% 1 Click <b>Create a new Access Key</b> to get a new Key & Secret pair for use with ODS-Briefcase: %BR%%BR%<a href="%ATTACHURLPATH%/step1-03.png" target="_blank"><img src="%ATTACHURLPATH%/step1-03.png" width="600px" /></a>%BR%%BR% 1 Click <b>Yes</b> when prompted: %BR%%BR%<a href="%ATTACHURLPATH%/step1-04.png" target="_blank"><img src="%ATTACHURLPATH%/step1-04.png" width="600px" /></a>%BR%%BR% 1 Your new Access Key ID will now be displayed. Click <b>Show</b> to see the corresponding Secret Access Key. %BR%%BR%<a href="%ATTACHURLPATH%/step1-05.png" target="_blank"><img src="%ATTACHURLPATH%/step1-05.png" width="600px" /></a>%BR%%BR% 1 You will need both of these for the steps that follow. %BR%%BR%<a href="%ATTACHURLPATH%/step1-06.png" target="_blank"><img src="%ATTACHURLPATH%/step1-06.png" width="600px" /></a>%BR%%BR% ---++++ 2 - Access files stored on Amazon S3 through ODS-Briefcase 1 Browse to your Virtuoso + ODS instance, <code>http://&lt;cname&gt;[:&lt;port&gt;]/ods</code>, e.g., &lt;[[http://example.com/ods/][http://example.com/ods/]]&gt;. %BR%%BR%<a href="%ATTACHURLPATH%/step2-01.png" target="_blank"><img src="%ATTACHURLPATH%/step2-01.png" width="600px" /></a>%BR%%BR% 1 Log in as a Virtuoso+ODS user with normal privileges, e.g., <code>demo</code>. %BR%%BR%<a href="%ATTACHURLPATH%/step2-02.png" target="_blank"><img src="%ATTACHURLPATH%/step2-02.png" width="600px" /></a>%BR%%BR% 1 Click to the <b>ODS-Briefcase</b> application: %BR%%BR%<a href="%ATTACHURLPATH%/step2-03.png" target="_blank"><img src="%ATTACHURLPATH%/step2-03.png" width="600px" /></a>%BR%%BR% 1 Click <b>New Folder</b> from the main Briefcase control strip: %BR%%BR%<a href="%ATTACHURLPATH%/step2-04.png" target="_blank"><img src="%ATTACHURLPATH%/step2-04.png" width="600px" /></a>%BR%%BR% 1 In the New Folder form, specify a <b>Folder Name</b>, e.g., <b><nop>MyAmazonS3Folder</b>, and set the <b>Folder type</b> to <b>Amazon S3</b>: %BR%%BR%<a href="%ATTACHURLPATH%/step2-05.png" target="_blank"><img src="%ATTACHURLPATH%/step2-05.png" width="600px" /></a>%BR%%BR% 1 Click to the newly visible <b>S3 Properties</b> tab: %BR%%BR%<a href="%ATTACHURLPATH%/step2-06.png" target="_blank"><img src="%ATTACHURLPATH%/step2-06.png" width="600px" /></a>%BR%%BR% 1 Set <b>Bucket Name</b> to a specific S3 Bucket Name to restrict ODS-Briefcase access to only this Bucket, or you can leave it empty to give ODS-Briefcase access to all your Buckets. Also enter the Access Key ID and Secret Key values you obtained earlier: %BR%%BR%<a href="%ATTACHURLPATH%/step2-07.png" target="_blank"><img src="%ATTACHURLPATH%/step2-07.png" width="600px" /></a>%BR%%BR% 1 All RDF data extracted from and otherwise collected about your S3 files will be saved to a named graph, where the default name is constructed from the host ODS instance, the ODS username, and the Briefcase folder name -- <code>http://&lt;host&gt;[:&lt;port&gt;]/DAV/home/&lt;user-name&gt;/&lt;folder-name&gt;#this</code> -- so, in this example, the default graph would be named &lt;[[http://example.com/ods/demo/MyAmazonS3Folder#this][http://example.com/ods/demo/MyAmazonS3Folder#this]]&gt;. Expert users may wish to change this graph name, but the default is suitable for most users. %BR%%BR%<a href="%ATTACHURLPATH%/step2-08.png" target="_blank"><img src="%ATTACHURLPATH%/step2-08.png" width="600px" /></a>%BR%%BR% 1 You can also turn on the [[http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSponger][Virtuoso Sponger]], to process your S3 files more comprehensively. When the Sponger is enabled for your folder, you can specify which "Sponger Extractor Cartridges" and "Sponger Meta Cartridges" will be run over the files in S3. You can enable all cartridges, but this will introduce some processing overhead. It's generally best to enable only those cartridges which will be applicable to your files and their content. For instance, you may know that you'll only be storing Microsoft and Open Office documents here. Click <b>Create</b> to finish the folder creation process. %BR%%BR%<a href="%ATTACHURLPATH%/step2-09.png" target="_blank"><img src="%ATTACHURLPATH%/step2-09.png" width="600px" /></a>%BR%%BR% 1 Your new S3-linked folder should now be displayed in your Briefcase resources list: %BR%%BR%<a href="%ATTACHURLPATH%/step2-10.png" target="_blank"><img src="%ATTACHURLPATH%/step2-10.png" width="600px" /></a>%BR%%BR% 1 Click the folder name to see the resources ODS-Briefcase has found in your Amazon S3 Dataspace: %BR%%BR%<a href="%ATTACHURLPATH%/step2-11.png" target="_blank"><img src="%ATTACHURLPATH%/step2-11.png" width="600px" /></a>%BR%%BR% ---+++References * [[http://virtuoso.openlinksw.com/wiki/main/Main/VirtuosoDAVS3][Virtuoso WebDAV S3 support]] * [[http://aws.amazon.com/s3/][Amazon S3 Services]] * [[http://docs.amazonwebservices.com/AmazonCloudFront/latest/GettingStartedGuide/][Amazon Cloud Front Getting Started Guide]] * [[http://bit.ly/SK7WAI][How Linked Data & WebID Deliver Controlled Access to Resources in Google Drive, Microsoft OneDrive (formerly SkyDrive), DropBox, Box.Net, and Amazon S3]] * [[OdsBriefcaseBoxNet][Using ODS-Briefcase to mediate access to data stored on Box (a/k/a box.com or box.net)]] * [[OdsBriefcaseDropBox][Using ODS-Briefcase to mediate access to data stored on DropBox]] * [[OdsBriefcaseGoogleDrive][Using ODS-Briefcase to mediate access to data stored on Google Drive]] * [[OdsBriefcaseSkyDrive][Using ODS-Briefcase to mediate access to data stored on Microsoft OneDrive (formerly SkyDrive)]] * [[OdsBriefcaseFolders][Manage different Briefcase type folders in Briefcase]] * [[OdsBriefcase][ODS-Briefcase]] * [[http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSponger][Virtuoso Sponger]] * [[http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSources][OpenLink-supplied Virtuoso Sponger Cartridges]] * [[http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMeta][Supported Meta Cartridges in Virtuoso]] * [[http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaREST][Meta Cartridges Usage via REST Request]] * [[http://virtuoso.openlinksw.com/dataspace/dav/wiki/Main/VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamples][Parameterized Examples of Meta Cartridge Usage via REST Request]] CategoryVirtuoso CategoryDAV CategoryTutorial CategoryDocumentation CategoryBriefcase CategoryODS CategoryOdsBriefcase

sioc:id

276f5600308c753151e5ded0c8dfeb72

sioc:link

n2:OdsBriefcaseAmazonS3

sioc:has_container

n19:ODS

n24:has_services

n25:item

atom:title

OdsBriefcaseAmazonS3

sioc:links_to

n2:CategoryODS n5:VirtSponger n2:CategoryDAV n5:VirtSpongerCartridgeSupportedDataSourcesMetaREST n5:VirtSpongerCartridgeSupportedDataSourcesMetaRESTExamples n12: n18: n2:CategoryVirtuoso n22:103-7399647-0537426 n23:html n2:OdsBriefcase n2:CategoryTutorial n2:CategoryOdsBriefcase n2:CategoryBriefcase n2:CategoryDocumentation n5:VirtSpongerCartridgeSupportedDataSources n2:OdsBriefcaseSkyDrive n5:VirtSpongerCartridgeSupportedDataSourcesMeta n40:VirtuosoDAVS3 n41:SK7WAI n42: n43:this n44:html n45:securityCredentials

atom:source

n19:ODS

atom:author

n27:this

atom:published

2017-06-13T06:01:46Z

atom:updated

2017-06-29T07:30:56Z 2018-01-02T10:53:36Z

sioc:topic

n19:ODS