Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Insert data #26

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
80 changes: 67 additions & 13 deletions walkthroughSteps/code-along-mentor-notes.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ git clone https://github.com/foundersandcoders/pg-walkthrough.git
## Step 1 – Navigating the initial files
1. Open `src/handler.js`.
- Here we see the `/static` endpoint reads and serves a file called `static.js`.

2. Open `src/static.js`
- We see that it contains a data array with two superhero objects.

Expand Down Expand Up @@ -41,7 +41,7 @@ git clone https://github.com/foundersandcoders/pg-walkthrough.git
weight INTEGER DEFAULT 100
);
```

- All tables should have an integer `id` that is set as a `PRIMARY KEY` - this is used relate databased together (integer PRIMARY KEY helps with performance)
- `PRIMARY KEY` also adds `UNIQUE` and `NOT NULL` (primary keys have to be unique).
- `VARCHAR(LENGTH)`, `INTEGER`, `TEXT` (unlimited length, but larger data usage), etc are SQL data types.
Expand All @@ -55,7 +55,7 @@ git clone https://github.com/foundersandcoders/pg-walkthrough.git
('Captain Marvel', 'Shoots concussive energy bursts from her hands', 165),
('Iron Man', 'None', 425);
```

- Rows separated with commas and each bracket, `(comma-separated values inside here)`, has a row inside it with values


Expand All @@ -78,13 +78,13 @@ Our database is now outlined, but we need a way to connect it

if (!process.env.DB_URL) throw new Error('Environment variable DB_URL must be set');
```

- `{ Pool }` is syntactic sugar (shorten/simplify syntax with abstraction) ([destructuring assignment](https://developer.mozilla.org/en/docs/Web/JavaScript/Reference/Operators/Destructuring_assignment)) that is equivalent to:
```js
const pg = require('pg');
const Pool = pg.Pool;
```

- [`Connection pooling`](https://en.wikipedia.org/wiki/Connection_pool) is a cache of multiple database connections that are kept open for a timeout period (`idleTimeoutMillis`) and reused when future requests are required, minimising the resource impact of opening/closing connections constantly for write/read heavy apps. Reusing connections minimises latency too. Debug/demo logging `Pool` might be helpful.
- `url` is a Node module - `url.parse()` will be used later
- You'll notice that this file requires a `config.env`. We'll set this up later.
Expand All @@ -96,14 +96,14 @@ Our database is now outlined, but we need a way to connect it
const params = url.parse(process.env.DB_URL);
const [username, password] = params.auth.split(':');
```

- `url.parse(<url string here>)` will split a URL/HREF string into an object of values like `protocol`, `auth`, `hostname`, `port`: [URL split documentation](https://nodejs.org/api/url.html#url_url_strings_and_url_objects)
- `[username, password]` is a ES6 destructuring assignment that is syntactic sugar for:
```js
const username = params.auth.split(':')[0];
const password = params.auth.split(':')[1];
```

Where username is index 0 of `params.auth.split(':')` and password is index 1, and so on.

5. Create a [`pg options`](https://node-postgres.com/features/connecting#programmatic) object:
Expand All @@ -118,7 +118,7 @@ Our database is now outlined, but we need a way to connect it
ssl: params.hostname !== 'localhost',
}
```

- Use an appropriate number for `max`. More connections mean more memory is used, and too many can crash the database server. Always return connections to the pool (don't block/freeze query callbacks), or the pool will deplete. More connections mean more queries can be run at once and more redundancy incase connections are blocked/frozen.
- `ssl` will enable SSL (set to true) if you're not testing on a local machine.
- TLS / SSL (Secure Sockets Layer) ensures that you are connecting to the database from a secure server, when set to `true`, preventing external networks from being able to read/manipulate database queries with MITM attacks
Expand All @@ -127,7 +127,7 @@ Our database is now outlined, but we need a way to connect it
```js
module.exports = new Pool(options);
```

- This exports the Pool constructor/object with the previously set options object, for other files to use this connection pool with `dbConnection.query` where `dbConnection` is the exported `Pool`.

7. Create a file: `database/db_build.js` with this code:
Expand All @@ -143,7 +143,7 @@ Our database is now outlined, but we need a way to connect it
console.log("Super heroes table created with result: ", res);
});
```

- Where `fs` is the Node file system module.
- `dbConnection` is the previously exported pool object.
- `sql` is a string of the build script. Think of it as a single query (transaction / collection of queries compiled into one).
Expand Down Expand Up @@ -177,7 +177,7 @@ Now that we have all the correct files, let's get this database up and running.
If you experience permission problems, try running `psql superheroes` then `GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO [the new username];`

## Step 5 – connecting to our database from the server
Let's first write a file that gets our information from the database.
- Let's first write a file that gets our information from the database.


1. Create a file `src/dynamic.js`.
Expand All @@ -197,7 +197,7 @@ Let's first write a file that gets our information from the database.
});
};
```

- If there's an error, return `cb(err)` - the return prevents the success code running.
- `res.rows` is an array of objects, where the objects are columns and values.

Expand All @@ -223,7 +223,61 @@ Let's first write a file that gets our information from the database.
response.end(dynamicData);
});
```

- `getData` is asynchronous, so `response.end` should be inside it, so it doesn't run before the data comes back from the database request (same as an API request).

7. Navigate to `http://localhost:3000/dynamic` to check it's worked.


- Now, let's write another function that inserts data into the database.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably be a 'Step 6' right?



1. Inside `src/dynamic.js`, write an asynchronous function `insertData` that takes a superhero information and a callback, and returns the callback function:
```js
const insertData = (superHeroInfo, cb) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There maybe needs to be some more explanation as to when this new function would be actually be called (i.e. in the handler for some post request, after the user fills out some form). And where the superHeroInfo comes from.

I guess it would expand the workshop too much to include actual functionality for making and handling a POST request, so I think there at least needs to be an explanation of why we are writing this function.

const { name, superPower, weight } = superHeroInfo;
dbConnection.query(`INSERT INTO superheroes (name, superPower, weight) VALUES (${name}, ${superPower}, ${weight})`, (err, res) => {
if (err) return cb(err)
return cb(null, res)
});
};
```

2. Export `insertData` along with the previous function `getData` we wrote:
```js
module.exports = { getData, insertData };
```

3. Into `src/handler.js`, import function `insertData` along with `getData` from `src/dynamic.js`:
```js
const { getData, insertData} = require('./dynamic');
```

4. Into a handler that recieves a post request use `insertData` to capture data sent from user and store into the database:
```js
// supposing a handler `postSuperHero` will handle this.
const superHeroInfo = req.body
insertData(superHeroInfo, (err, res) => {
if (err) return console.log(err);
const insertResult = JSON.stringify(res);
response.writeHead(200, { 'content-type': 'application/json' });
response.end(insertResult);
});
```

5. Navigate `http://localhost:3000/post-superhero` to Check your insert result.

6. Security Vulnerabilities:

Inserting into database as we did is a security issue, and to prevent SQL Injections, we can replace the previous query we used with this:
```js
// at function insertData(superHeroInfo, cb) ::
const secured = {
text: `INSERT INTO superheroes (name, superPower, weight) VALUES ($1, $2, $3)`,
values: [name, superPower, weight]
};
dbConnection.query(secured, (err, res) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does passing this secured object work?

I would usually do dbConnection.query(text, values, (err, res) => { })

as per the docs: https://node-postgres.com/features/queries#parameterized-query

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah you can pass a query config object with text and value properties. That might be confusing to students who have only ever seen values passed a the second argument though

// block of code.....
}

```
10 changes: 9 additions & 1 deletion walkthroughSteps/step-5/src/dynamic.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,12 @@ const getData = (cb) =>{
});
};

module.exports = getData;
const insertData = (superHeroInfo, cb) => {
const { name, superPower, weight } = superHeroInfo;
dbConnection.query(`INSERT INTO superheroes (name, superPower, weight) VALUES (${name}, ${superPower}, ${weight})`, (err, res) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the solution should be the 'safe' version I think

if (err) return cb(err)
return cb(null, res)
});
};

module.exports = { getData, insertData };