Separate v1 and v2

.gitignore

@@ -7,3 +7,4 @@ _pdf
 .DS_Store
 temp
 Gemfile.lock
+_docs/assets/architecture/.structurizr

_config.yml

@@ -123,7 +123,7 @@ defaults:
 sidebars:
 - sidebar_ioos
 
-description: "A Jekyll-based documentation theme for data management documentation, software and other technical documentation needs.  This is a skeleton repository for IOOS technical documentation projects to use as a starting point to develop new pages for the https://ioos.github.io site."
+description: "Technical documentation for the IOOS HFRNet System"
 # the description is used in the feed.xml file
 
 #disqus_shortname: idrbwjekyll

_data/sidebars/sidebar_ioos.yml

@@ -19,14 +19,14 @@ entries:
       output: web
 
 # Top-level menu item #1
-    - title: Homepage Header 1
+    - title: HFRNet v1.0
 
 # A link to the 1st rendered document; consists of '/'+'markdown-document-name'+'.html'[+'#'+'any-header-or-other-anchor-on-the-page'
-      url: /index.html#header-1
+      url: /hfrnetv1p0.html
       output: web
 
-    - title: Homepage Header 2
-      url: /index.html#header-2
+    - title: HFRNet v2.0
+      url: /hfrnetv2p0.html
       output: web
 
       subfolders:

_docs/01-context.md

@@ -0,0 +1,17 @@
+---
+# Feel free to add content and custom Front Matter to this file.
+# To modify the layout, see https://jekyllrb.com/docs/themes/#overriding-theme-defaults
+
+title: Context
+layout: home
+---
+
+The HFRNet system is the information management system that collects raw data from ~165 High Frequency Coastla Radar sites around the country and processes it into national products that serve various [users](./users) needs.  
+
+- What is this software project/product/system all about?
+- What is it that’s being built?
+- How does it fit into the existing environment? (e.g. systems, business processes, etc) -  Who is using it? (users, roles, actors, personas, etc)
+
+TODO update and insert context diagram with HFRNet and main external software systems.  Probably worth describing the difference between software systems that serve as a conduit to users, and the end users themselves.  For example, NWTSG -> GTS -> global NMHS.  Or WSTG -> AWIPS (tools used by WFOs).
+
+![High level context diagram](portals.svg)

_docs/02-functional-overview.md

@@ -0,0 +1,43 @@
+---
+# Feel free to add content and custom Front Matter to this file.
+# To modify the layout, see https://jekyllrb.com/docs/themes/#overriding-theme-defaults
+
+title: Functional Overview
+layout: home
+---
+
+
+
+[FDD](./hfrnet-functional-decomposition)
+
+Even though the purpose of a software guidebook isn’t to explain what the software does in detail, it can be useful to expand on the context and summarise what the major functions of the software are.
+Intent
+This section allows you to summarise what the key functions of the system are. It also allows you to make an explicit link between the functional aspects of the system (use cases, user stories, etc) and, if they are significant to the architecture, to explain why. A functional overview should answer the following types of questions:
+
+- Is it clear what the system actually does?
+- Is it clear which features, functions, use cases, user stories, etc are significant to the
+architecture and why?
+- Is it clear who the important users are (roles, actors, personas, etc) and how the system
+caters for their needs?
+- It is clear that the above has been used to shape and define the architecture?
+Alternatively, if your software automates a business process or workflow, a functional view should answer questions like the following:
+- Is it clear what the system does from a process perspective?
+- What are the major processes and flows of information through the system?
+
+## Structure
+
+By all means refer to existing documentation if it’s available; and by this I mean functional specifications, use case documents or even lists of user stories. However, it’s often useful to summarise the business domain and the functionality provided by the system. Again, diagrams can help, and you could use a UML use case diagram or a collection of simple wireframes showing the important parts of the user interface. Either way, remember that
+the purpose of this section is to provide an overview.
+Alternatively, if your software automates a business process or workflow, you could use a flow chart or UML activity diagram to show the smaller steps within the process and how they fit together. This is particularly useful to highlight aspects such as parallelism, concurrency, where processes fork or join, etc.
+
+## Motivation
+
+This doesn’t necessarily need to be a long section, with diagrams being used to provide an overview. Where a context section summarises how the software fits into the existing environment, this section describes what the system actually does. Again, this is about providing a summary and setting the scene rather than comprehensively describing every user/system interaction.
+
+## Audience
+
+Technical and non-technical people, inside and outside of the immediate software develop- ment team.
+
+## Required
+
+Yes, all software guidebooks should include a summary of the functionality provided by the software.

_docs/03-quality-attributes.md

@@ -0,0 +1,52 @@
+---
+# Feel free to add content and custom Front Matter to this file.
+# To modify the layout, see https://jekyllrb.com/docs/themes/#overriding-theme-defaults
+
+title: Quality Attributes
+layout: home
+---
+
+## 3. Quality Attributes
+
+With the functional overview section summarising the functionality, it’s also worth includ- ing a separate section to summarise the quality attributes/non-functional requirements.
+Intent
+This section is about summarising the key quality attributes and should answer the following types of questions:
+
+- Is there a clear understanding of the quality attributes that the architecture must satisfy?
+- Are the quality attributes SMART (specific, measurable, achievable, relevant and timely)?
+- Havequalityattributesthatareusuallytakenforgrantedbeenexplicitlymarkedasout of scope if they are not needed? (e.g. “user interface elements will only be presented in English” to indicate that multi-language support is not explicitly catered for)
+- Areanyofthequalityattributesunrealistic?(e.g.true24x7availabilityistypicallyvery costly to implement inside many organisations)
+In addition, if any of the quality attributes are deemed as “architecturally significant” and therefore influence the architecture, why not make a note of them so that you can refer back to them later in the document.
+Structure
+Simply listing out each of the quality attributes is a good starting point. Examples include:
+- Performance (e.g. latency and throughput)
+- Scalability (e.g. data and traffic volumes)
+- Availability (e.g. uptime, downtime, scheduled maintenance, 24x7, 99.9%, etc) -  Security (e.g. authentication, authorisation, data confidentiality, etc)
+- Extensibility
+- Flexibility
+- Auditing
+- Monitoring and management -  Reliability
+- Failover/disaster recovery targets (e.g. manual vs automatic, how long will this take?) -  Business continuity
+- Interoperability
+- Legal, compliance and regulatory requirements (e.g. data protection act)
+- Internationalisation (i18n) and localisation (L10n) -  Accessibility
+- Usability
+- ...
+
+Each quality attribute should be precise, leaving no interpretation to the reader. Examples where this isn’t the case include:
+
+- “the request must be serviced quickly” -  “there should be no overhead”
+- “as fast as possible”
+- “as small as possible”
+- “as many customers as possible” - ...
+
+## Motivation
+
+If you’ve been a good software architecture citizen and have proactively considered the quality attributes, why not write them down too? Typically, quality attributes are not given to you on a plate and an amount of exploration and refinement is usually needed to come up with a list of them. Put simply, writing down the quality attributes removes any ambiguity both now and during maintenance/enhancement work in the future.
+
+## Audience
+
+Since quality attributes are mostly technical in nature, this section is really targeted at technical people in the software development team.
+
+## Required
+Yes, all software guidebooks should include a summary of the quality attributes/non- functional requirements as they usually shape the resulting software architecture in some way.

_docs/04-constraints.md

@@ -0,0 +1,49 @@
+---
+# Feel free to add content and custom Front Matter to this file.
+# To modify the layout, see https://jekyllrb.com/docs/themes/#overriding-theme-defaults
+
+title: Constraints
+layout: home
+---
+
+# 4. Constraints
+
+Software lives within the context of the real-world, and the real-world has constraints. This section allows you to state these constraints so it’s clear that you are working within them and obvious how they affect your architecture decisions.
+
+## Intent
+
+Constraints are typically imposed upon you but they aren’t necessarily “bad”, as reducing the number of available options often makes your job designing software easier. This section allows you to explicitly summarise the constraints that you’re working within and the decisions that have already been made for you.
+
+## Structure
+
+As with the quality attributes, simply listing the known constraints and briefly summarising them will work. Example constraints include:
+
+- Time, budget and resources.
+- Approved technology lists and technology constraints.
+- Target deployment platform.
+- Existing systems and integration standards.
+- Local standards (e.g. development, coding, etc).
+- Public standards (e.g. HTTP, SOAP, XML, XML Schema, WSDL, etc). -  Standard protocols.
+- Standard message formats.
+- Size of the software development team.
+- Skill profile of the software development team.
+- Nature of the software being built (e.g. tactical or strategic).
+- Political constraints.
+- Use of internal intellectual property.
+- etc
+
+If constraints do have an impact, it’s worth summarising them (e.g. what they are, why they are being imposed and who is imposing them) and stating how they are significant to your architecture.
+
+## Motivation
+
+Constraints have the power to massively influence the architecture, particularly if they limit the technology that can be used to build the solution. Documenting them prevents you having to answer questions in the future about why you’ve seemingly made some odd decisions.
+
+## Audience
+
+The audience for this section includes everybody involved with the software development process, since some constraints are technical and some aren’t.
+
+## Required
+
+Yes, all software guidebooks should include a summary of the constraints as they usually shape the resulting software architecture in some way. It’s worth making these constraints explicit at all times, even in environments that have a very well known set of constraints (e.g. “all of our software is ASP.NET against a SQL Server database”) because constraints have a habit of changing over time.
+
+Ideally this project serves as a pathfinder for building out other elements of DMAC in a cloud hosted way. Need to think harder about the benefits of doing this.  Is it worth it?  What are the benefits of having tighter integration of DMAC services in NESDIS Operations?

_docs/05-principles.md

@@ -0,0 +1,15 @@
+---
+# Feel free to add content and custom Front Matter to this file.
+# To modify the layout, see https://jekyllrb.com/docs/themes/#overriding-theme-defaults
+
+title: Principles
+layout: home
+---
+
+
+
+- Multi-stage migration
+- Change the fewest number of system components as possible during each stage of the migration.
+- Open source code for all system functions 
+- Open project planning to encourage transparency and invite contributions. (Create ioos/hfrnet github repository to serve as the master repo for planning and version control.  OK to create new repos to facilitate a modular and open architecture.  e.g. separate "science code" from "synchronizing and monitoring data flow")
+- Provide access to all data types, not just the end product.

_docs/06-software-architecture.md

@@ -0,0 +1,52 @@
+---
+# Feel free to add content and custom Front Matter to this file.
+# To modify the layout, see https://jekyllrb.com/docs/themes/#overriding-theme-defaults
+
+title: Software Architecture
+layout: home
+---
+
+The software architecture section is your “big picture” view and allows you to present the structure of the software. Traditional software architecture documents typically refer to this as a “conceptual view” or “logical view”, and there is often confusion about whether such views should refer to implementation details such as technology choices.
+
+## Intent
+
+The purpose of this section is to summarise the software architecture of your software system so that the following questions can be answered:
+
+- What does the “big picture” look like?
+- Is there are clear structure?
+- Is it clear how the system works from the “30,000 foot view”?
+- Does it show the major containers and technology choices?
+- Does it show the major components and their interactions?
+- Whatarethekeyinternalinterfaces?(e.g.awebservicebetweenyourwebandbusiness
+tiers)
+
+## Structure
+
+I use the container and component diagrams as the main focus for this section, accompanied by a short narrative explaining what the diagram is showing plus a summary of each container/component.
+Sometimes UML sequence or collaboration diagrams showing component interactions can be a useful way to illustrate how the software satisfies the major use cases/user stories/etc. Only do this if it adds value though and resist the temptation to describe how every use case/user story works!
+
+## Motivation
+
+The motivation for writing this section is that it provides the maps that people can use to get an overview of the software and help developers navigate the codebase.
+
+## Audience
+
+The audience for this section is predominantly the technical people in the software develop- ment team.
+
+## Required
+
+Yes, all software guidebooks should include a software architecture section because it’s essential that the overall software structure is well understood by everybody on the development team.
+
+## Ingesting data from HFR sites into the HFRNet System
+
+Two options.  Parallel development
+
+### Option 1: Use existing portals 
+
+See item 0001 in the [Decision Log](./13-decision-log) for more details.
+
+![Schematic of existing portal data flow](./portals.svg)
+
+### Option 2: Ingest from sites directly into the cloud
+
+![Schematic showing direct ingest into AWS S3 buckets](./cloud.svg)

_docs/07-code.md

@@ -0,0 +1,49 @@
+---
+# Feel free to add content and custom Front Matter to this file.
+# To modify the layout, see https://jekyllrb.com/docs/themes/#overriding-theme-defaults
+
+title: Code
+layout: home
+---
+
+
+Although other sections of the software guidebook describe the overall architecture of the software, often you’ll want to present lower level details to explain how things work. This is what the code section is for. Some software architecture documentation templates call this the “implementation view” or the “development view”.
+
+## Intent
+
+The purpose of the code section is to describe the implementation details for parts of the software system that are important, complex, significant, etc. For example, I’ve written about the following for software projects that I’ve been involved in:
+
+- Generating/rendering HTML: a short description of an in-house framework that was created for generating HTML, including the major classes and concepts.
+- Data binding: our approach to updating business objects as the result of HTTP POST requests.
+- Multi-page data collection: a short description of an in-house framework we used for building forms that spanned multiple web pages.
+- Web MVC: an example usage of the web MVC framework that was being used.
+- Security:ourapproachtousingWindowsIdentityFoundation(WIF)forauthentication
+and authorisation.
+- Domain model: an overview of the important parts of the domain model.
+- Component framework: a short description of the framework that we built to allow
+components to be reconfigured at runtime.
+- Configuration: a short description of the standard component configuration mecha-
+nism in use across the codebase.
+- Architectural layering: an overview of the layering strategy and the patterns in use to
+implement it.
+- Exceptionsandlogging:asummaryofourapproachtoexceptionhandlingandlogging
+across the various architectural layers.
+- Patterns and principles: an explanation of how patterns and principles are imple-
+mented.
+- etc
+
+## Structure
+
+Keep it simple, with a short section for each element that you want to describe and include diagrams if they help the reader. For example, a high-level UML class and/or sequence diagram can be useful to help explain how a bespoke in-house framework works. Resist the temptation to include all of the detail though, and don’t feel that your diagrams need to show everything. I prefer to spend a few minutes sketching out a high-level UML class diagram that shows selected (important) attributes and methods rather than using the complex diagrams that can be generated automatically from your codebase with UML tools or IDE plugins. Keeping any diagrams at a high-level of detail means that they’re less volatile and remain up to date for longer because they can tolerate small changes to the code and yet remain valid.
+
+## Motivation
+
+The motivation for writing this section is to ensure that everybody understands how the important/significant/complex parts of the software system work so that they can maintain, enhance and extend them in a consistent and coherent manner. This section also helps new members of the team get up to speed quickly.
+
+## Audience
+
+The audience for this section is predominantly the technical people in the software develop- ment team.
+
+## Required
+
+No, but I usually include this section for anything other than a trivial software system.

_docs/08-data.md

@@ -0,0 +1,40 @@
+---
+# Feel free to add content and custom Front Matter to this file.
+# To modify the layout, see https://jekyllrb.com/docs/themes/#overriding-theme-defaults
+
+title: Data
+layout: home
+---
+
+# 8. Data
+
+The data associated with a software system is usually not the primary point of focus yet it’s arguably more important than the software itself, so often it’s useful to document something about it.
+
+## Intent
+
+The purpose of the data section is to record anything that is important from a data perspective, answering the following types of questions:
+
+- What does the data model look like?
+- Where is data stored?
+- Who owns the data?
+- How much storage space is needed for the data? (e.g. especially if you’re dealing with
+“big data”)
+- What are the archiving and back-up strategies?
+- Are there any regulatory requirements for the long term archival of business data? -  Likewise for log files and audit trails?
+- Are flat files being used for storage? If so, what format is being used?
+
+## Structure
+
+Keep it simple, with a short section for each element that you want to describe and include domain models or entity relationship diagrams if they help the reader. As with my advice for including class diagrams in the code section, keep any diagrams at a high level of abstraction rather than including every field and property. If people need this type of information, they can find it in the code or database (for example).
+
+## Motivation
+
+The motivation for writing this section is that the data in most software systems tends to outlive the software. This section can help anybody that needs to maintain and support the data on an ongoing basis, plus anybody that needs to extract reports or undertake business intelligence activities on the data. This section can also serve as a starting point for when the software system is inevitably rewritten in the future.
+
+## Audience
+
+The audience for this section is predominantly the technical people in the software develop- ment team along with others that may help deploy, support and operate the software system.
+
+## Required
+
+No, but I usually include this section for anything other than a trivial software system.