Skip to content

Commit

Permalink
init
Browse files Browse the repository at this point in the history
  • Loading branch information
@healtho
healtho committed Aug 13, 2019
1 parent 617e2ae commit 56214ea
Show file tree
Hide file tree
Showing 15 changed files with 1,239 additions and 0 deletions.
62 changes: 62 additions & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
##########################
## Java
##########################
*.class
.mtj.tmp/
*.jar
*.war
*.ear
hs_err_pid*
target/

##########################
## Maven
##########################
target/
pom.xml.tag
pom.xml.releaseBackup
pom.xml.versionsBackup
pom.xml.next
release.properties

##########################
## IntelliJ
##########################
*.iml
.idea/
*.ipr
*.iws
out/
.idea_modules/

##########################
## Eclipse
##########################
.metadata
.classpath
.project
.settings/
bin/
tmp/
*.tmp
*.bak
*.swp
*~.nib
local.properties
.loadpath

##########################
## NetBeans
##########################
nbproject/private/
build/
nbbuild/
dist/
nbdist/
nbactions.xml
nb-configuration.xml

##########################
## OS X
##########################
.DS_Store
163 changes: 163 additions & 0 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@

## Use JDBC driver to query Rest APIs

This is a simple project which explains the usage of [Apache Calcite](https://calcite.apache.org/) to query [NSE](https://www.nseindia.com/live_market/dynaContent/live_watch/equities_stock_watch.htm) data APIs using JDBC interface. The similar concepts could be used to query any rest API.

For detailed explanation, read [blog]()

### How to run

To run the project, clone it in a folder and make sure to have below pre-requisites installed and setup:

* JAVA 1.8 or above
* Maven 3.6 or above

**Build the project**

```shell
mvn clean install -DskipTests
```

**Run [Sqlline](https://github.com/julianhyde/sqlline)**

A command line utility for issuing SQL to relational databases via JDBC.

```shell
./sqlline
```

*if using Windows OS, then use `sqlline.bat` file*

This would open a sqlline shell, which can now be used to connect to JDBC driver and issue SQL queries

*image here*

**Create a model.json file**

Define a model.json file, which should define the schema and the schema factory class, as:

```json
{
"version": "1.0",
"defaultSchema": "BroadMarket",
"schemas": [
{
"name": "BroadMarket",
"type": "custom",
"factory": "org.gitcloned.calcite.adapter.nse.NSESchemaFactory",
"operand": {
"group": "Broad Market Indices"
}
}
]
}
```

**Connect to JDBC using model.json**

```shell
!connect jdbc:calcite:model=/path/to/model.json admin admin
```

This will connect to the JDBC schema defined in model.json file, to list the tables run `!tables` command, which will list all tables along with some system tables:

```sql
+-----------+-------------+---------------------------------+--------------+---+

| TABLE_CAT | TABLE_SCHEM | TABLE_NAME | TABLE_TYPE | R |

+-----------+-------------+---------------------------------+--------------+---+

| | BroadMarket | NIFTY JUNIOR | TABLE | |

| | BroadMarket | NIFTY MIDCAP 150 | TABLE | |

| | BroadMarket | NIFTY MIDCAP 50 | TABLE | |

| | BroadMarket | NIFTY SMLCAP 250 | TABLE | |

| | BroadMarket | NIFTY SMLCAP 50 | TABLE | |

| | BroadMarket | NIFTY50 | TABLE | |

| | Others | SOVEREIGN GOLD BONDS | TABLE | |

| | Sectoral | NIFTY AUTO | TABLE | |

| | Sectoral | NIFTY BANK | TABLE | |

| | Sectoral | NIFTY ENERGY | TABLE | |

| | Sectoral | NIFTY FINANCIAL SERVICES | TABLE | |

| | Sectoral | NIFTY FMCG | TABLE | |

| | Sectoral | NIFTY IT | TABLE | |

| | Sectoral | NIFTY METAL | TABLE | |

| | Sectoral | NIFTY PHARMA | TABLE | |

| | Sectoral | NIFTY REALTY | TABLE | |

| | Strategy | NIFTY DIVIDEND OPPORTUNITIES 50 | TABLE | |

| | Thematic | NIFTY COMMODITIES | TABLE | |

| | metadata | COLUMNS | SYSTEM TABLE | |

| | metadata | TABLES | SYSTEM TABLE | |

+-----------+-------------+---------------------------------+--------------+---+
```

To list available columns, run `!columns` command:

```sql
+-----------+-------------+---------------------------------+------------------+

| TABLE_CAT | TABLE_SCHEM | TABLE_NAME | COLUMN_NAME |

+-----------+-------------+---------------------------------+------------------+

| | Strategy | NIFTY DIVIDEND OPPORTUNITIES 50 | trdVolM |

| | Strategy | NIFTY DIVIDEND OPPORTUNITIES 50 | wkhi |

| | Strategy | NIFTY DIVIDEND OPPORTUNITIES 50 | wklo |

| | Strategy | NIFTY DIVIDEND OPPORTUNITIES 50 | yPC |

| | Strategy | NIFTY DIVIDEND OPPORTUNITIES 50 | time |

| | Thematic | NIFTY COMMODITIES | symbol |

| | Thematic | NIFTY COMMODITIES | open |

| | Thematic | NIFTY COMMODITIES | high |

| | Thematic | NIFTY COMMODITIES | low |

| | Thematic | NIFTY COMMODITIES | previousClose |

| | Thematic | NIFTY COMMODITIES | ltp |

| | Thematic | NIFTY COMMODITIES | per |

| | Thematic | NIFTY COMMODITIES | trdVolM |

| | Thematic | NIFTY COMMODITIES | wkhi |

| | Thematic | NIFTY COMMODITIES | wklo |

| | Thematic | NIFTY COMMODITIES | yPC |

| | Thematic | NIFTY COMMODITIES | time |
+-----------+-------------+---------------------------------+------------------+
```

Next, try SQL queries over data

### Other Details

* Log File: application.log
129 changes: 129 additions & 0 deletions pom.xml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,129 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>org.gitcloned.calcite</groupId>
<artifactId>caclite-rest-example</artifactId>
<version>1.0-SNAPSHOT</version>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
<configuration>
<source>7</source>
<target>7</target>
</configuration>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>clirr-maven-plugin</artifactId>
<version>2.8</version>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>3.0.0</version>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>javacc-maven-plugin</artifactId>
<version>2.4</version>
</plugin>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>build-helper-maven-plugin</artifactId>
<version>3.0.0</version>
<executions>
<execution>
<id>add-resource</id>
<phase>generate-resources</phase>
<goals>
<goal>add-resource</goal>
<goal>add-test-resource</goal>
</goals>
<configuration>
<resources>
<resource>
<directory>D:\work\OpenSource\CalciteRestAPIExample</directory>
<targetPath>META-INF</targetPath>
<includes>
<include>LICENSE</include>
<include>NOTICE</include>
</includes>
</resource>
<resource>
<directory>D:\work\OpenSource\CalciteRestAPIExample/target</directory>
<targetPath>META-INF</targetPath>
<includes>
<include>git.properties</include>
</includes>
</resource>
</resources>
</configuration>
</execution>
<execution>
<id>add-test-sources</id>
<phase>generate-test-sources</phase>
<goals>
<goal>add-test-source</goal>
</goals>
<configuration>
<sources>
<source>D:\work\OpenSource\CalciteRestAPIExample\target/generated-test-sources/javacc</source>
</sources>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>

<properties>
<build.timestamp>${maven.build.timestamp}</build.timestamp>
<calcite.version>1.20.0</calcite.version>
<avatica.version>1.15.0</avatica.version>
</properties>

<dependencies>
<dependency>
<groupId>org.apache.calcite.avatica</groupId>
<artifactId>avatica</artifactId>
<version>${avatica.version}</version>
</dependency>
<dependency>
<groupId>org.apache.calcite</groupId>
<artifactId>calcite-core</artifactId>
<version>${calcite.version}</version>
</dependency>
<dependency>
<groupId>com.google.http-client</groupId>
<artifactId>google-http-client</artifactId>
<version>1.30.2</version>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.5</version>
</dependency>
<dependency>
<groupId>sqlline</groupId>
<artifactId>sqlline</artifactId>
<version>1.8.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.25</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.25</version>
</dependency>
</dependencies>
</project>
Loading

0 comments on commit 56214ea

Please sign in to comment.