Luigi - Job Monitoring Server

Luigi is a Python (2.7, 3.3, 3.4, 3.5) package that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization, handling failures, command line integration, and much more.

Installing Luigi

Step 1 >> pip install luigi (command line)

Step 2 >> luigid (command line)

Step 3 >> Browse localhost:8082

Google BigQuery: Convert Date Time to YYYYMMDD format

The below syntax helps to convert date time format to date format using Google BigQuery:

SELECT
<date_time column> DATETIME, STRFTIME_UTC_USEC(<date_time column>,'%Y%m%d')
as DATE
FROM <table>