Help Needed - Ingesting data from Slack API into Snowflake Datawarehouse

Hi there - I am new to snowflake and wanted to post here to see if anybody has experience in ingesting data from slack api into snowflake Datawarehouse. If so please do let me know the steps and details with an example or so on how that can be achieved. Thanks in advance,

Lakshman

nekkanti009

posted on 29 Oct 19

Enjoy great content like this and a lot more !

Signup for a free account to write a post / comment / upvote posts. Its simple and takes less than 5 seconds

Post Comment

nVector30-Oct-19

There are lot of automated prebuilt pipelines in the market. But from the question, i understand you want to build your own.

How does the slack api send your data, If its in JSON format you can directly insert it into a Snowflake table. You will have to write a custom code (python / java / ...) that polls the slack api at regular intervals to retrieve the records from last read

ReplyMore..ReportDelete

nekkanti00930-Oct-19

Thank you npack for the reply. Yes, the slack api will send the data in JSON format. Oh ok.. Do you happen to have any sample python code by any chance? If so please share. I will try looking in that direction to write python code.

ReplyMore..ReportDelete

nVector31-Oct-19

Unfortunately, I do not have a code sample. Ill try to put a code skeleton if i find some time. At a high level, below would be your next steps:

1.First start by using an API examiner using free tools like postman / soapui. They let you call apis and also show the response from the server. This exercise will help you visualize how it works end to end

2.As a second step, you will write your custom script that calls the api and inserts the json response to a table.

ReplyMore..ReportDelete

nekkanti00931-Oct-19

Sounds good, will try that. Thank you.

ReplyMore..ReportDelete