Test create-probot-app against this template

[tcbyrd]

As discussed in #75, because of the handlebars template variables we use in package.json, the normal npm test command doesn't work in Travis. This PR will update the .travis.yml file to run create-probot-app in Travis, then run the tests in the cloned template. We'll need to ensure Travis scaffolds the repo from the right branch, so I'm thinking in addition to this we add a command-line flag to create-probot-app that allows you to specify a branch of the template repo(which is an option available in scaffold). We would then pass the ${TRAVIS_PULL_REQUEST_BRANCH} environment variable to this script.

The goal is to ensure any changes to this template can successfully scaffold an app with all the available options and run a clean npm test.

[tcbyrd]

@probot/maintainers Would ❤️to hear feedback on this approach. Test are passing, so if I'm thinking through this correctly, any PRs to the template will have to successfully scaffold an app in Travis and run that app's tests to result in a successful build.

[bkeepers]

Keep in mind this file will be used as the template for all new Probot apps.

What if we modify the way create-probot-app looks for templates, and instead of just using the repository that is passed to it, we look for a template directory within the repository? This would allow the root of the repository to be support files (e.g. a better readme that explains this repo is a template, and proper tests for the template).

[hiimbex]

What if we modify the way create-probot-app looks for templates, and instead of just using the repository that is passed to it, we look for a template directory within the repository? This would allow the root of the repository to be support files (e.g. a better readme that explains this repo is a template, and proper tests for the template).

That would be ideal, it would also make it easier to maintain the usecase examples. Do you think that would be doable @tcbyrd ?

[tcbyrd]

Just to make sure I'm understanding correctly, the repo will be fully functional and testable but have a template folder that create-probot-app scaffolds from?

So for a single template, maybe something like this?

template
|-- src/
| .  |-- index.js
|-- test/
| .  |-- index.test.js
|-- template/
| .  |-- test/
| .       |-- index.test.js
| .  |-- .env.example
| .  |-- index.js
| .  |-- package.json.ejs
| .  |-- README.md.ejs
|-- .gitignore
|-- .travis.yml
|-- package.json

Another option is we use this repo as a single source for all our templates. This could be one way to structure that (borrowing slightly from oclif/templates)
_{It's after 1am as I'm writing this, so it may be a crazy idea. 😪}

templates
|-- base/   # common files for all templates
| .  |-- .env.example
| .  |-- LICENSE
| .  |-- CONTRIBUTING.MD
| .  |-- CODE-OF-CONDUCT.MD
|-- checks/
| .  |-- index.js
| .  |-- index.test.js
| .  |-- package.json.ejs
| .  |-- README.md.ejs
|-- hello-world/
| .  |-- index.js
| .  |-- index.test.js
| .  |-- package.json.ejs
| .  |-- README.md.ejs
|-- .gitignore
|-- .travis.yml
|-- package.json

I see pros and cons for both. I'm leaning towards multiple templates in the same repo because I tend to like how that helps with discovery in other frameworks (e.g. koa/examples) and I feel like it would easier to manage in one repo. Then in create-probot-app we make template=checks scaffold from the base/ and related checks/ folder.

[bkeepers]

@tcbyrd I disabled Travis CI against this repo until this PR is merged.

[bkeepers]

Submitting a review to remove this from my review queue.

[hiimbex]

Also submitting a review to hit inbox 0 for pull reminders! I think this can probably be closed in favor of the work in probot/create-probot-app#36 which will include something similar to this