keboola/magento-extractor-bundle

Some new component

# Magento Extractor configuration

Authentication

POST/GET on /oauth endpoint Must contain token and config parameters, where config is a table(must already exist) name of the particular configuration. The config table must have api_url, oauth.consumer_secret and oauth.consumer_key attributes set prior to requesting the authentication., _(*1)

Config table

Attributes

• api_url: The base URL of your application - http://magentohost in the Magento documentation, eg. http://yourapplication.com (without /api/rest)
• oauth.consumer_key: Consumer key generated in Magento Admin UI - see Admin documentation
• oauth.consumer_secret: Consumer secret obtained in the same fashion as oauth.consumer_key

Remaining necessary OAuth information generated by /oauth endpoint, _(*2)

Data

• Columns:
  • endpoint(required): The API endpoint
  • params: Query parameters of the api call, JSON encoded
    • Each parameter in the JSON encoded object may either contain a string, eg: {""key"": ""value""}
    • Filters can be configured by setting two parameters as seen in the example below:
      • "filter[0][attribute]":"created_at"
      • "filter[0][gt]":"2014-12-09 22:02:53"
    • User functions are supported with metadata containing data from sys.c-ex-magento-metadata.{config} table (see the example data below)
  • dataType: Type of data returned by the endpoint. It also describes a table name, where the results will be stored
  • dataField: Allows to override which field of the response will be exported. Only used with data API
    • If there's multiple arrays in the response "root" the extractor may not know which array to export and fail
    • If the response is an array, the whole response is used by default
    • If there's no array within the root, the path to response data must be specified in dataField
    • Can contain a path to nested value, dot separater (eg result.results.products)
  • rowId(required): An unique identificator of the configuration row

Example data

"endpoint","params","dataType","dataField","recursionParams","rowId"
"orders","{
""limit"":100,
""filter[0][attribute]"":""created_at"",
""filter[0][gt]"":{
    ""function"": ""date"",
    ""args"": [
      ""Y-m-d H:i:s"",
      {
        ""function"": ""strtotime"",
        ""args"": [{""metadata"":""jobs.lastStart.orders""}]
      }
    ]
  }
}","","","","orders"
"customers","{""limit"":100}","","","","customers"

Group API

Extractor run [/run]

Run extraction [POST]

JSON Parameters:, _(*3)

• config (required) ... configuration id (name of configuration table), _(*4)

• Request (application/json), _(*5)

  • Headers, _(*6)

    Accept: application/json
    X-StorageApi-Token: Your-Sapi-Token

  • Body, _(*7)

    {
        "config": "main"
    }

  • Schema, _(*8)

    {
        "type": "object",
        "required": true,
        "properties": {
            "config": {
                "type": "string",
                "required": true
            }
        }
    }

• Response 201 (application/json), _(*9)

  {
      "id": "48419532",
      "url": "https://syrup.keboola.com/queue/job/48419532",
      "status": "waiting"
  }

Generate OAuth token [/oauth{?token,config}]

Generate token from a web form/UI [POST]

• Parameters, _(*10)

  • token = `` (required, string, 305-78945-rg48re4g86g48gwgr48e6) ... Your KBC Token, _(*11)

  • config = `` (required, string, main) ... Config table name / configuration ID, _(*12)

• Request (multipart/form-data; boundary=----WebKitFormBoundaryC5GD12ZfR1D8yZIt), _(*13)

  • Body, _(*14)

    ------WebKitFormBoundaryC5GD12ZfR1D8yZIt
    Content-Disposition: form-data; name="token"
    
    305-78954-d54f6ew4f84ew6f48ewq4f684q
    ------WebKitFormBoundaryC5GD12ZfR1D8yZIt--
    
    ------WebKitFormBoundaryC5GD12ZfR1D8yZIt
    Content-Disposition: form-data; name="config"
    
    main
    ------WebKitFormBoundaryC5GD12ZfR1D8yZIt--

  • Schema, _(*15)

    {
        "type": "object",
        "required": true,
        "properties": {
            "config": {
                "type": "string",
                "required": true
            }
            "token": {
                "type": "string",
                "required": true
            }
        }
    }

• Response 201 (application/json), _(*16)

  {
      "status": "ok"
  }

Generate token manually [GET]

• Parameters, _(*17)

  • token = `` (required, string, 305-78945-rg48re4g86g48gwgr48e6) ... Your KBC Token, _(*18)

  • config = `` (required, string, main) ... Config table name / configuration ID, _(*19)

• Response 201 (application/json), _(*20)

  {
      "status": "ok",
      "oauth_token": "dsjioafhoiy832yt598y7895y",
      "oauth_token_secret": "kf98v0894u8j580jy8902xyjciurewc"
  }