Notes.txt

New Requirements - Use can select to read from web or from file based on user input

(Start with Test Shells first)

Step 1 - Move (alg) Logic to .WordCounting
Step 2 - Setup Http 
    --- no binding for WebClient
Step 3 Create WordCountingEngine ties all the pieces together
    --- ties all the SR pieces together
Step 4 Create UI.Repl
Step 5 Note how we can't create a Repl, implementation of IWordCountEngine is internal, can't access.  Temporary give access now.
Step 6 Update Main().  Highlight manual creation of object graph (poor man DI)
Step 7 Run
Step 8 Write initial unit test
 -- Use [TestCase] to extract input arguments from Test Case
 -- Use SkyKick.Bcl.Extensions to read Embedded Resources
 -- Use MockRepository to create a dynamic proxy object 
 that allows us a number of powerful options.  We can stub
 out fake behaviors, inspect method arguments and a lot more.
 This is your entry point for creating this mocked objects.  It's 
 possible to use concrete objects that have virtual methods, but its
 a hell of a lot easier to use interfaces.  Which is one of the 
 reasons why its good practice to create an interface, 
 even if you will only have one implementation.
 -- Use .Stub (as opposed to .Expect) - provide behavior,
    but don't need to validate it's called
 -- Arg.Is() mostly convienance here, but also validates
    input is passed in correctly
 -- Should library has conienance methods for Asserts

 CheckPoint 1
 -- Introduce Ninject (replaces building object graph)
 Step 9 Create Kernel in UI
     - Add Statup class with a Build Kernel method.  
	    This should NOT be static.
     Kernel should only be accessed from an Entry Point
	 (RoleEntryPoint for Cloud Services, Main in Console
	 Web Apps use a plugin that injects kernel into the 
	 Controller Factory, so you never access it directly)
	 - Call BuildKernel in main

Step 10 Run Program - Note Activation Exception.  Can't find
    IWordCountingEngine.  Because we haven't added a binding for it yet.

Step 11 Add Ninject Module for .WordCounting
        Highlight using NonPublicTypes.  There
		should be no reason to leak concrete implementations
		other than to the Test class that's explicitly testing
		these types.
		Remove .UI from InternalsVisibleTo
        Update BuildKernel()
		   Highlight using full namespace

Step 12 Run Program - Note different Activation Exception  (IWebClient)

Step 13 Lets TDD this problem.  
       Create a Test proving Bindings
	   Verify failure

Step 14 Manual Binding
		WebClientWrapper does not match default naming convention,
		add manual binding

CheckPoint 2

Step 15 Run Bindings Test
     Note, we've made progress, but still getting Activation Exception()
	 But we're closer - missing a binding for SkyKick.Bcl.Logging

Step 16 Adding binding for SkyKick.Bcl.Logging
     we'll add the NinjectModule that comes with that library

Step 17 Run Bindings Test - Confirm that test passes.

Step 18 Run UI - Confirm application works

Step 19 Clean up WordCountingEngineTests
     We can use a kernel instead of creating
	 WordCountingEngine manually
	 - Show use of .ReBind() - ninject alreadys has a binding, 
	 we need to replace it.
	 - .ToContant allows us to bind to an existing instance rather
	 than a type that Ninject will control creating.
	 - Highlihgt importance of .BuildKernel creating a new instance
	 every time.  Because we are manipulating bindings we need
	 to make sure we have a fresh instance, otherwise we could be polluting
	 other tests, or other tests could have polluted us by chaning bindings.
	 - Confirm test works

Check Point 3

The core prototype "works", has been converted to use SOLID principles and
we have 86% code coverage of .WordCounting.

But the core algorithm is not very good.  Lets prove it's not as robust as
it could be by adding a failing a test.  This is TDD style, we've found a bug
so first lets verify we can reproduce the bad behavior

   Step 20 Add WordsWithEntersAndNoSpaces
       Be sure to mark as embedded resource

   Step 21 Add new input file to CountsWordsInSampleFilesCorrectly
       Verify test fails

	Step 22  Update Word Counting Algorithm - very simple/naive fix

	Step 23 Re run CountsWordsInSampleFilesCorrectly and verify bug is fixed

Check Point 4

We are missing cross cutting concerns.  Lets add some in and see how they 
can be tested.

Lets add the requirement that if the IWebClient throws a general exception or 
gets a 500, we should retry 3 times with a back off period of 0.5s, 1s and 10s.

   Step 24  Add Polly to .WordCounting

   Step 25 Update WebTextSource 
      - We could add retry to WebClientWrapper, but we want wrappers
	    to be very light weight, they really shouldn't include any 
	    additional logic ontop of the api code they wrap.
	  - Note how WebTextSourceOptions is injected.  This means
	    WebTextSource is not responsible for knowing how to get its
		own settings, it must be injected.  This also gives us great flexiblity
		for testing.
	  - Its ok for WebTextSourceOptions to include a default
	  - This pattern aligns very nicely with SkyKick.Bcl.Configuration
	    which provides a DI supported subsytem for configuration
	  - Note: on the ExecuteAsync lambda, the _ for the lambda parameter.
	    This is short hand indicating that the variable (cancellation token)
		wont be used.

	Step 26 Add WebTextSourceTests
	  - Normally it would be very hard to test a retry policy based 
	    on an exception thrown by a 3rd party/framework utility, but
		because we have a wrapper and WebTextSourceOptions, it's quite easy
	  - Use [TestCaseSouce] to point to a method that generates test input,
	    allowing us to run code to generate Test Cases, wouldn't be possible
		with just [TestCase].  This allows our test code to test a single hypothesis
		(specific eception throws error) while still maximizing code reuse
	  - Use .Expect() to have the ability to Verify that method was called
	    with given method parameters a set number of times.
	  - Use .Thorw() to easily have a mock throw an exception
	  - We create a WebTextSourceOptions with an array of 0 second retry times
	    to Verify() that the retry policy is retrying Web Requests
	  - Use VerifyAllExpectations() to verify GetHtmlAsync was called the correct
	    number of times

	Step 27 Run Unit Tests
	   - Note two tests pass, but the test where we did NOT expect the
	     retry behavior to be triggered failed.
		 --- We found a bug in the retry logic - it retries on 
		     a non-transient exception.  That would have been very very
		     hard to identify in a running system!
       - The Exception that is logged is quiet daunting.  We caught
	     an exception, but it's not the exception we thought it would be, 
		 so the ShouldEqaul(webClientException) threw a new exception. 
		 The "Actual" exception is what was thrown by the WebTextSource:
		 A NullReferenceException.
		    --- This is a very important exception to understand when
			working with Mocks, especially when dealing with Async code.
			--- Key to understanding is knowing how a Mock behaves by default,
			    which is it will return default() for any method that has 
				not been stubbed with either Stub() or Expect().  When we
				an Async method is called on a Mock with no Stub, Rhino
				will return null, and the code will end up trying to `await null`
				which leades to the NullReferenceException.

Check Point 5

	Step 28 Strict Mock
		- Improve InvokesRetryPolicyOnErrors by changing 
		  GenerateMock() to GenerateStrictMock()
		- Rerun the failing test
		- We now get a better Exception in the Actual output a
		  ExpectationViolationException.  Using Strict mocks will
		  have Rhino throw a very specific Exception if the code 
		  under test tries to invoke a method that hasn't been stubbed.
		  This is quite useful for helping to diagnose failing tests
		  that use mocks, and is something I'm working on using more myself.

	Step 29 Fix WebTextSource
	    - Update .Or<Exception> to exclude handling a generic exception
		  if it is a WebException: .Or<Exception>(ex => !(ex is WebException)) 
		
	Step 30 Run All Tests
		- Verify that you just diagnosed and fixed a retry policy bug
		  completly in unit tests, before your code ever made it to prod!
		
Check Point 6

Performance optimization time.  We expect are Word Counter will be asked to count
the same url over and over again.  So to speed performance, we'll add a cache. 
But there's a catch, the cache we will use has a start up penalty.  Before we'd
use the Singleton pattern to make sure we only instantiated one instance of the 
cache so we'd only get hit with the penalty once.  But we can use Ninject 
to have only one instance of the cache and still support DI and not use statics!

     Step 31 Create Cache
		- Note how we use IThreadSleeper to wrap the call to Thread.Sleep.  While
		  this might seem a bit extereme, it's very helpful in enabling us to write
		  a unit test that doesn't rely on a call try TryGet() taking a long time.

	Step 32 Add Cache to WordCountingEngine

	Step 33 Run WordCounting.UI
		- Enter https://www.skykick.com.  Note the log message
		  that the Cache is initializing and the program waits for 3 seconds.
		- Enter https://www.skykick.com again.  Note how there is no log
		  message about initialization and instead we get a log message about a 
		  cache hit.
		- The .UI program is not running multithreaded and the way it's designed,
		  the Repl class keeps the full object graph between user input so it's
		  ok that WordCountCache is not actually a singleton.

	Step 34 Guard Test
		- Event though WordCounting.UI isn't using the cache from multiple 
		  requests, .WordCounting might need to support more advanced scenarios
		  in the future, so we want to document that it should be created as a 
		  Singleton.  We'll create a Guard Test - a quick test that protects a 
		  small but very important implementation detail
		- Because we are testing a component of .WordCounting, it's not really
		  appropriate or necessary to use UI.Startup().BuildKernel(), so we'll 
		  create a new one, using only the modules necessary to build WordCountCache.
		- Note that when stubbing a method that has optional parameters, it's always
		  necessary to pass Arg values, otherwise RhinoMocks will throw an exception.
		- We can Bind mocks to a StandardKernel for our test and Ninject is perfectly 
		  happy.
		  --- However, for IThreadSleeper we must use Rebind().  the .WordCounting 
		      NinjectModule already has a binding for IThreadSleeepr.  If we use
			  Bind<IThreadSleeper>.ToConstant(mockThreadSleeper) the call will succeed,
			  however when we do a kernel.Get Ninject will throw an exception because 
			  it will not know which of the two bindings to use.  
		  --- There is no problem if you use Rebind if there is not an existing binding.
		- Not how it's very useful to have a wraper around Thread.Sleep, it allows
		  the test to run in a fraction of a second, instead of waiting three seconds
		  for the Initialize methods to complete.
		- Because we're using quantum logging that supports DI, we can also verify
		  that logging occurs :)
		- Run the test, confirm that it fails.  Ninject is exhibiting default behavior,
		  each call to kernel.Get<IWordCountCache>() will return a new instance

	Step 35 Singleton Scope
		- Update Ninject Module
		- The InSingletonScope instructs Ninject to only create one instance of
		  a class on the first request and then reuse it for all subsequent requests.
		- Notice how we have to use Rebind in this case, because the 
		  SelectAllClasses().BindDefaultInterface() will include a default binding
		  for IWordCountCache.

	Step 36 Run Guard Test
		- Confirm the test now passes!
		- An interesting thing to note, is we only set InSingletonScope() when
		  IWordCountCache is requested.  If you were to change the test to request
		  WordCountCache it would again fail because Ninject would create two 
		  different instances for the request to Get<WordCountCache>().
		- This can be fixed by adding Bind<WordCountCache>().ToSelf().InSingletonScope()
		  in the Ninjet Module.
		  --- Note the use of .ToSelf(), this is done instead of 
		      Bind<WordCountCache>().To<WordCountCache>()
		  --- That fixes if both requests are for Get<WordCountCache>().  But what if one
		      request was Get<IWordCountCache>() and the other was Get<WordCountCache>()?
			  Then it would fail, because Ninject sees it as two different requests with
			  different InSingletonScopes().  To solve that is certainly possible, but 
			  requires more advanced bindings:

			     Kernel.Bind<WordCountCache>().To<WordCountCache>().InSingletonScope();
                 Kernel.Rebind<IWordCountCache>().ToMethod(ctx => ctx.Kernel.Get<WordCountCache>());

		  --- When would you use this?  It's valuable when use interface segregation but have
		      one object implement two interfaces.  For example, if you had seperate interfaces
			  for a repository, one read only and one write only: IUserReadRepository and 
			  IUserWriteRepository and both interfaces are implemented by UserRepository.  If
			  UserRepository needed to be a Singleton because it did some long running 
			  initialization, then it would be necessary to use this technique to make sure
			  a request to either interface returned the same instance:
			    
		         Kernel.Bind<UserRepository>().To<UserRepository>().InSingletonScope();
                 Kernel.Bind<IUserReadRepository>().ToMethod(ctx => ctx.Kernel.Get<UserRepository>());
				 Kernel.Bind<IUserWriteRepository>().ToMethod(ctx => ctx.Kernel.Get<UserRepository>());

	Step 37 Fix Cross Component Tests
		- You might have noticed that our cross component tests are now running
		  a lot longer - WordCountingEngine is having to initialize its cache.
		- Add a mock IThreadSleeper that doesn't actually sleep so our tests run 
		  quickly again.
		- This is another benefit of having the IThreadSleeper wrapper
				 
Check Point 7

New Requirements - Read from File This will require a bit of a redesign
as the initial design was tightly coupled with the idea of reading from Web pages.

	Step 38 Create ITextSource
		- To make 'text source' generic, we can't have a named method
		  that takes initialization data.  We'll need to initialize in the constructor
		- Expose a TextSourceId for logging / cache key

	Step 39 Update WebTextSource
		- Take url in the constructor
		--- Now we have a parameter that we need to pass in to the constructor that 
		    does not support DI.  Time to use a Factory

	Step 40 Define WebTextSourceFactory
	    - We'll need to create an interface to define the factory signature.  
		- Implementation will use constructor injection to pull in all fo the 
		  dependencies that WebTextSource needs, and then will complement that with 
		  the non-injectable parameters need (url)
		- This allows us to still use DI everywhere, but still support initialization
		  input that will be provided by run time data; in this case user input
	
	Step 41 Update WordCountingEngine to use ITextSource
		- Replaces url parameter
		- (General refactoring)

	Step 42  Create FileTextSource / Factory
		- We'll use SkyKick.Bcl.Extensions.File.IFile to pull in an existing
		  abstraction around the File System
		- For IFileTextSourceFactory we'll use a plugin to avoid having
		  to write the boiler plate factory code that pulls in the dependencies
		  and passes them to the FileTextSouce constructor
		 --- This use a number of convention. Method must start with Create and we must
		     create a IFileTextSource to help the Factory

	Step 43 Update NinjectModule with Factory Binding
		- Add nuget package

	Step 44 Move Repl to its own namespace
		- We're going to create a few more files, so lets organize

	Step 45 Create ReplTextSourceBuilder
		- This will drive accepting user input and using the correct 
		  Text Source Builder
		- We inject both factories and then decide, based on user input, 
		  which one to use to build the ITextSource we want to build

	Step 46 Update Repl to use ReplTextSourceBuilder

	Step 47 .UI Ninject Module
		- We're now injecting a IReplTextSourceBuilder into
		  Repl.  We don't have a Ninject Module for .UI so Repl
		  will no longer resolve correclty.
		- Create a new default Ninject Module for .UI and add to Startup.BuildKernel()

	Step 48 Run All Tests to verify everything still works

Check Point 8

Arbitrary Complexity using WordCountingWorkflow
   (Send Email 1 if words > 1000, else send email 2)
   (Save results to Word Count History Table)

   Step 49 Create a dummy Email Client

   Step 50 Create WordCountingWorkflow

   Step 51 Update Repl to use WordCountingWorkflow

   Step 52 Verify all Tests pass after refactor

   Step 53 Create WordCountingWorkflowTests
		- WordCountingWorkflow has a lot of dependencies,
		  so we'll use a Mocking Kernel to make it easier to deal with them.
		  Mocking Kernel will automatically mock any dependency that is requested.
		  We can also add real bindings to it if we wanted.
		- Install nuget package.  NOTE: You must update Ninject.MockingKernel or you'll
		  get a nasty error when running the test.
		- Use the .Get<>().Stub() to directly add a Stub to a mock.
		- Note we haven't added any behavior for ILogger, mocking kernel will take care
		  of it for us, we don't have to do anything for it.
		- Pro Tip - I like to add hints in test description on how to interpret and fix
		  a test if it shows failure conditions.  In this case, if the wrong email is sent,
		  a null refernec exception will be thrown, so I doucment this in the test comments

   Step 54 Intro to BDD
		- Uses syntax Given, When, Then to describe setup for a scerario, execution of a scenario, and then
		  all of the exepctations following execution.
		- Idea is to build up a library of components that make it very easy to get the system to a point
		  where we can perform the test and then have a shared list of validations we can perform.  This enables up
		  to rapidly add more scenarios as well as provide a framework for describing and validating complex
		  logics.
		- Very similar to a Cross Componenet test, the idea here is to test as much of the stack as possible.
		  So we'll only mock out the WebClient and EmailClient, so we're fully running WordCountingWorkflow,
		  WordCountingEngine, WordCountingAlgorithm, WordCountCache, and WebTextSource	
		- This is still a technique that I'm learnign, so this is the 'best practice' that I've learned thus far.
		  Additionally, we want to standardize this process by using SpecFlow which is a framework for creating tests
		  in this style.  It creates method stubs for much of what's currently in TestHarness and then provdies 
		  a plain text style DSL for stiching together the methods that appear in the Test Fixtures
		- End result of this is a very human readable list of Business 
		  Scenarios that can be perfomed on a System and a list of results
		  that should occur when the Scneario is executed.
		- Create WordCountingWorkflowScenarioTests file

	Step 55 Create Test Harness
		- This will be used by all the Scenarios
		- Fluent Syntax, just something I like to do, method chaining is easy to read IMO
		- Highlight .Do on mocking Logger.Debug so we can still see messages
		- Test Harness sets up Mocks and then exposes helper methods for our BDD tests to perform setup
		  and validation.

	Step 56 Create Scenarios
		- Add new sample files 500words, 3000words
		- We can create a 'happy path' flow with 500 and 3000 words
		- We can also validate error scenarios, verifying behavior of the system
		  when there is a Exception

Check Point 9

Plugins


   -------

   Why Service Locator is bad
    - Using DI makes it very clear which dependencies are being used.  DI declaratively
	  lists dependencies for a class. Service locator, the class does declare any
	  dependencies, it pulls in whatever it wants.
	- With DI at the app entry point, the kernel builds the full object graph.  With
	  Service Locator, the graph is constantly being rebuilt at every Get request
	- Classes shouldn't be responsible for knowing where to get there dependencies 
	  (ie Service Locator).  They should just delcare what they need and not worry how
	  dependencies are satisfied.  Externalize (invert) dependency resolution.