Software Crafters Lyon
Tu ne le sais pas encore, mais tu l’as deja documenté!
Marc Bouvier • u.baldir.fr/me
Software Crafters Lyon
Software Crafters Strasbourg
Marc Bouvier u.baldir.fr/me
Technical Coach
Software Developer
Learning and promoting
Accessibility
Green IT
Crédit photo : Joël Hellenbrand
Cette présentation contient des démos foireuses
Es-tu satisfait de la documentation que tu produis?
Quand tu lis de la documentation, est-ce que tu as peur qu’elle soit obsolète?
Quand tu utilises des composants tiers est-ce que tu aimerais que leur documentation soit meilleure?
Est-ce que tu penses que le temps passé à écrire de la documentation pourrait être mieux utilisé?
The Story of an onboarding
Living Documentation FTW
A ton of examples!
Some Takeaways
Joining an hypothetical organisation as a consultant
Meet with colleagues
Fetch laptop
You have E-mail 📧
🎉
No Git credentials… yet
"It will take some time"
Go explore "The Wiki"
A shared folder with MS Office files
A ticket for MS Office Licence
Browsing "The Wiki"
A typo
Cannot edit page
"Just add a comment to it, we’ll fix it later"
Spoiler: it wasn’t
No license available
Human PDF converter 👩🏻💻🖨
Wrong version
"Nobody really reads it anyway"
Windows XP IE9 screenshots… 🦖
.. in 2020
Maybe a little outdated
Finally Git credentials!
Tech stack evolved a lot since spec
Versions bumped
Even framework is not the same
It’s about..
Vistors
Printers
Factories
Strategies?…
Aren’t we in banking business?
Design Patterns?
Naming Convention
"Technical separation: Tidy!"
A reason?
"We do it everywhere"
Where is the business
"Inside the subfolders scattered around"
Must be hard to test?
"Well, we tried, but they were always broken"
We do End 2 End but they are flaky
We hired someone full-time to maintain them
"Worth the effort, it tests some behavior"
Business complaints about human readability
Click
Forms
Fields
What are the features?
"I do caching related stuff, and some DB related stuff"
But Business features
"We have user stories in Jira tickets, it’s all there"
…
Newcomers are great to see what’s missing
Usually external documents
Not in sync with the system
Versioning hell
Specs 👨💼
Code 👩💻
️Tests 🕵️
Doc 👩🏫
Even in Agile teams!
A pain to update
Requires special tooling
Boring manual transcription
No one really reads it cover to cover
Not easy to engage conversations with
What the system should have been
When it was written
Often by a different person
🧐
Sharing knowledge
Knowledge needed for a long period of time?
→ Documented
Knowledge needed by a large number of people
→ Documented
Knowledge valuable or critical
→ Documented
Default
→ Don’t
Programming languages
Developer tools
Software practices
DDD
patterns
CI
Mature sector of the business industries
Pricing in finance
Supply Chain Optimization in e-commerce
→ Industry standard books
The Generic Knowledge is already documented in the industry literature.
→ Learn Generic Knowledge
→ Link to sources of Authority
Training
Conferences
Books
Blog posts
Standard Vocabulary
…
Not (yet) shared with other peers in the same industry.
More expensive to learn
That takes practice, making mistakes to earn
That Deserves more attention
Only you can take care about it
Stable
Standards
Most of parts of common industry
Not Stable
Marketing
Company names
Dates
People
Design decisions
Learn & refer to it
Automate it as much as possible when it is useful
Internal
Code
Tests
Manifests
Wiki as code
External
External Wiki
Ms Office documents
Project management tools
Not accessible
To many (noise)
Scattered everywhere
Implicit (e.g. design pattern visible only if we know it exists)
Unrecoverable (lost)
Unwritten
Augment it, make it accessible for it’s audience.
Useful content for a long period of time
Short with not much details
High Level focus
Goals and intention (not implementaion details)
More business oriented that technical
Here traditional documentation is OK 👍
File names
Classes, functions, variables names
Folder arborescence
Automated tests
File names
Tests names
Tests methods names
Tests datasets
Conversations
On their machine
Javadoc
Comments → website
Comments → contextual doc in IDE
Xml documentation
Comments → website
Comments → contextual doc in IDE
Generators: DocFX, Sandcastle, Doxygen
Jsdoc
Comments → website
Comments → contextual doc in IDE
Java : Maven / Gradle
Js/TS : package.json
Dotnet : .csproj .fsproj or .vbproj
docker-compose.yml
Helm charts
Terraform
Ansible
.gitlab-ci.yml
Github Actions
UX
Help section
Tooltips
Exceptions & errors traces
Database logs
Ticket / Bugs backlog
git history
Task management tools activity
Improving Onboarding Experience 🛥
Understanding systems and improving them
Communicating better (less misunderstandings) 🗣
To build the right things
Your team’s morale
Against Truck factor anxiety 🚛
Reduce Turnover rate
Reduce Bugs because of knowledge ignorance
Reducing Time writing documentation
Reducing time to look for something in documentation
Reducing context switching …
Reliable
Low-Effort
Collaborative
Insightful
Exploit available knowledge
In code, artifacts, tools
Accuracy mechanism
To make sure knowledge is up to date
Simplicity
Standard over custom
Evergreen content
Refactoring proof Knowledge
Internal Documentation
Favors conversations over formal documentation
Collective Ownership
Deliberate decision making
Embedded learning
Reality check
e.g. pep8, zen of python
import this
Repeated informations
package org.mycompany.worlddestroyer;
/**
* The implementation of the Death Star. (1)
* @author Joe@empire.org (2)
*/
public class DeathStarImpl (3)
implements DeathStarInterface (4)
{}
Tells nothing more than the class signature
In internal projet, maybe rely on version control history
Of course it is an implementation
Of course it is an interface
Ubiquitous Language
Code that reads like prose
Fluent interfaces
You don’t need more than your testing lib
But some features can help
Nesting
Parameterized tests
class Agile_tour_speaker_should{
@Test
void share_screen_to_streamyard(){
given_a_speaker_ready_to_speak();
when_the_talk_has_started();
then_Philippe_is_not_yelling_a_the_speaker();
}
}
Pick the style you want but make sure it is readable by non technical people
Feature: Agile Tour speaker
Scenario: Share screen to streaming service
Given A speaker ready to speak.
When The talk has started
Then Philippe is not yelling at the speaker
Only Business language (no technical)
Pickles
Cucumber studio
Specflow+ LivingDoc
…
Test REST API
Endpoint documentation as test code
Snippet Generation
Python Doctest
Elixir Doctest
Rust Documentation test
Go Test Examples
def sum(a, b):
"""
Adds two numbers.
>>> sum(2,2)
5
"""
return a + b
if __name__ == "__main__":
import doctest
doctest.testmod()
***********************************************************
File ".../python_doctest/sum.py", line 5, in __main__.sum
Failed example:
sum(2,2)
Expected:
5
Got:
4
***********************************************************
1 items had failures:
1 of 1 in __main__.sum
***Test Failed*** 1 failures.
defmodule Num do
@moduledoc """
Operations on numbers
"""
@doc """
Adds 2 numbers
## Examples
iex> Num.sum(2, 2)
5
"""
def sum(a, b) do
a + b
end
end
mix test
Compiling 1 file (.ex)
1) doctest Num.sum/2 (1) (NumTest)
test/num_test.exs:3
Doctest failed
doctest:
iex> Num.sum(2, 2)
5
code: Num.sum(2, 2) === 5
left: 4
right: 5
stacktrace:
lib/num.ex:11: Num (module)
Finished in 0.02 seconds (0.00s async, 0.02s sync)
1 doctest, 1 failure
/// Adds two numbers together:
/// ```
/// # use documentation_test_rust::add;
/// let c = add(2, 2);
/// # assert_eq!(c, 5);
/// ```
pub fn add(a: i32, b: i32) -> i32 {
return a + b;
}
Doc-tests documentation_test_rust
running 1 test
test src/lib.rs - add (line 2) ... FAILED
failures:
---- src/lib.rs - add (line 2) stdout ----
Test executable failed (exit code 101).
stderr:
thread 'main' panicked at 'assertion failed: `(left == right)`
left: `4`,
right: `5`', src/lib.rs:6:1
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
failures:
src/lib.rs - add (line 2)
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.18s
error: test failed, to rerun pass '--doc'
package num
func Sum(a int, b int) int {
return a + b
}
package num
import "fmt"
func ExampleSum() {
fmt.Println(Sum(2, 2))
// Output: 5
}
--- FAIL: ExampleSum (0.00s)
got:
4
want:
5
FAIL
exit status 1
FAIL doctest_golang 0.303s
Javadoc @snippet
tag
package fr.baldir.samples.hello;
public class Hello {
/**
* Greets someone.
* <p>Usage</p>
* {@snippet class="snippets.SnippetTest" region="test-region"}
*
* @param name target of greetings
* @return full greeting sentence
*/
public String sayHello(String name) {
return "Hello " + name;
}
}
package snippets;
import fr.baldir.samples.hello.Hello;
import org.assertj.core.api.Assertions;
import org.junit.jupiter.api.Test;
public class SnippetTest {
@Test
public void someTest() {
// @start region="test-region"
Hello hello = new Hello();
var greetings = hello.sayHello("JEP 413");
// @end
Assertions.assertThat(greetings).isEqualTo("Hello JEP 413");
}
}
layeredArchitecture()
.layer("Controller").definedBy("..controller..")
.layer("Service").definedBy("..service..")
.layer("Persistence").definedBy("..persistence..")
.whereLayer("Controller").mayNotBeAccessedByAnyLayer()
.whereLayer("Service").mayOnlyBeAccessedByLayers("Controller")
.whereLayer("Persistence").mayOnlyBeAccessedByLayers("Service")
From your code
Do you see mainly domain related names?
Or technical related names?
Code metadata
Easily searchable Grepable by tools
Conversation starter (better than preaching)
@GoodExample
@BadExample
Generated from code
Make Business Knowledge accessible
Makes knowledge mistmatch visible
Leads to Ubiquitous Language
Java files with marker annotations (ex. @Aggregate
, @Entity
).
Inspiration for such annotations are shown in Patternity by Cyrille Martraire.
ShoppingCart.java
package com.organisation.domain.order;
/**
* A shopping cart blah blah blah.
*/
@Aggregate
public class ShoppingCart{
// ...
}
ShoppingCartItem.java
package com.organisation.domain.order;
/**
* An item that is hold the shopping cart to be ordered.
*/
@Entity
public class ShoppingCartItem{
// ...
}
# Order
[Link](/src/main/java/com/organisation/domain/order/)
## Shopping Cart
A shopping cart blah blah blah.
[Link](/src/main/java/com/organisation/domain/order/ShoppingCart.java)
### Shopping Cart Item
An item that is hold the shopping cart to be ordered.
[Link](/src/main/java/com/organisation/domain/order/ShoppingCartItem.java)
PlantUml
MermaidJs
Ditaa
Graphviz
…
@startuml
[*] --> State1
State1 --> [*]
State1 : this is a string
State1 : this is another string
State1 -> State2
State2 --> [*]
@enduml
+-------------+
| Asciidoctor |-------+
| diagram | |
+-------------+ | PNG out
^ |
| ditaa in |
| v
+--------+ +--------+----+ /---------------\
| | --+ Asciidoctor +--> | |
| Text | +-------------+ | Beautiful |
|Document| | !magic! | | Output |
| {d}| | | | |
+---+----+ +-------------+ \---------------/
: ^
| Lots of work |
+-----------------------------------+
May be supported in some tools
Previews in IDE
Gitlab/Github/Azure DevOps supports mermaidjs
Most of the time focus on content and not layout
It’s plain text
Easy to version
Can be generated from code
Static code Analysis
Sonarlint / Sonarqube / Sonarcloud
Linters
IDEs
CI/CD Tools
Code
Tooling configuration
Aggregated Data
Visualizations
Hints, Warning
Learning material
Source Code
.gitlab-ci.yml
configuration
include:
- template: "Security/SAST.gitlab-ci.yml"
- template: "Security/Dependency-Scanning.gitlab-ci.yml"
- template: "Security/Secret-Detection.gitlab-ci.yml"
[background-color="#3942b0"] 🥷 === Gitlab Security Analyzers Output
Pipeline report artifacts
Json files, Generated Html
Security Dashboard
CodeTour VsCode extension
Source Code
Json File (machine-readable)
{
"$schema": "https://aka.ms/codetour-schema",
"title": "Frontend basic tour",
"steps": [
{
"file": "js/public/index.html",
"description": "## Html entry point : `public/index.html`\n\nThis `public/index.html` is the first file that is loaded by the browser. \nIt loads the bundled VueJs application as a script.\nThe rendering of the application is then delegated to javascript.\n\nThe VueJs app is \"injected\" inside the `<div is=\"app\"></div>` element.\n\nThe bundled javascript script will be inserted in the `index.html` file in place of the comment : `<!-- built files will be auto injected -->`",
"line": 19,
"selection": {
"start": {
"line": 20,
"character": 5
},
"end": {
"line": 20,
"character": 47
}
},
"contents": "<!DOCTYPE html>\n<html lang=\"en\">\n <head>\n <meta charset=\"utf-8\" />\n <meta http-equiv=\"X-UA-Compatible\" content=\"IE=edge\" />\n <meta name=\"viewport\" content=\"width=device-width,initial-scale=1.0\" />\n <link rel=\"icon\" href=\"<%= BASE_URL %>favicon.ico\" />\n <meta name=\"server-injected-data\" />\n </head>\n\n <body>\n <noscript>\n <strong\n >We're sorry but <%= htmlWebpackPlugin.options.title %> doesn't work\n properly without JavaScript enabled. Please enable it to\n continue.</strong\n >\n </noscript>\n <div id=\"app\"></div>\n <!-- built files will be auto injected -->\n </body>\n</html>\n"
}
],
"isPrimary": true,
"description": "A quick overview of how browsing to the default URI translate to VueJs components and behaviours",
"ref": "09cc5d1bd1ecd24e1e879b0392cf5830b2581c74"
}
Interactive Tour in Vs Code
Custom Annotations (Meta-data)
@GuidedTour(1)
Comments + Asciidoc
Mix markup and code and you obtain interactive notebooks
Mixed Markdown and code
Combines both
UI component showcase
Executable Technical Documentation
Clever Cloud UI Components
Systeme de design du gouvernement français
this
...
...
Included code snippets are real sources
AsciiDoctor
RevealJs
include::../examples/doctest/python_doctest/sum.py
Doctor Who - BBC
Good commit messages
Tells the WHY
The template
<type>[optional scope]: <description>
[optional body]
[optional footer(s)]
Example 1
feat: allow provided config object to extend other configs
BREAKING CHANGE: `extends` key in config file is now used for extending other config files
Example 2
docs: correct spelling of CHANGELOG
Conventional Changelog
is the new @author
Published as Static site
Architecture Decisions Records
Markdown / Asciidoc / PLain text
Templates
helpers tools
# Use Markdown Architectural Decision Records
- Status: accepted
- Date: 2020-09-24
## Context and Problem Statement
We want to record architectural decisions made in this project.
Which format and structure should these records follow?
## Considered Options
- [MADR](https://adr.github.io/madr/) 2.1.2 with Log4brains patch
- [MADR](https://adr.github.io/madr/) 2.1.2 – The original Markdown Architectural Decision Records
- [Michael Nygard's template](http://thinkrelevance.com/blog/2011/11/15/documenting-architecture-decisions) – The first incarnation of the term "ADR"
- [Sustainable Architectural Decisions](https://www.infoq.com/articles/sustainable-architectural-design-decisions) – The Y-Statements
- Other templates listed at <https://github.com/joelparkerhenderson/architecture_decision_record>
- Formless – No conventions for file format and structure
## Decision Outcome
Chosen option: "MADR 2.1.2 with Log4brains patch", because
- Implicit assumptions should be made explicit.
Design documentation is important to enable people understanding the decisions later on.
See also [A rational design process: How and why to fake it](https://doi.org/10.1109/TSE.1986.6312940).
- The MADR format is lean and fits our development style.
- The MADR structure is comprehensible and facilitates usage & maintenance.
- The MADR project is vivid.
- Version 2.1.2 is the latest one available when starting to document ADRs.
- The Log4brains patch adds more features, like tags.
The "Log4brains patch" performs the following modifications to the original template:
- Change the ADR filenames format (`NNN-adr-name` becomes `YYYYMMDD-adr-name`), to avoid conflicts during Git merges.
- Add a `draft` status, to enable collaborative writing.
- Add a `Tags` field.
Gitlab/Github Markdown renderer
Published Site
Human readable plain text files
Excel files exchanges from emails
Versioning hell
Hard to show progress from Excel in meetings
Standardization of content (CSV)
Versionning with Source control
Publication as website
During audit: Incremental feedback, support conversations
After audit: Ready to be published
CSV
Yaml
Markdown
CSV file
Could be sourced from ADRs
Google sheet
Generated static site
version: 3.8
services:
backend:
build:
args:
- NODE_ENV=development
context: backend
command: npm run start-watch
environment:
- DATABASE_DB=example
- DATABASE_USER=root
- DATABASE_PASSWORD=/run/secrets/db-password
- DATABASE_HOST=db
- NODE_ENV=development
ports:
- 80:80
- 9229:9229
- 9230:9230
secrets:
- db-password
volumes:
- ./backend/src:/code/src:ro
- ./backend/package.json:/code/package.json
- ./backend/package-lock.json:/code/package-lock.json
- back-notused:/opt/app/node_modules
networks:
- public
- private
depends_on:
- db
db:
# We use a mariadb image which supports both amd64 & arm64 architecture
image: mariadb:10.6.4-focal
# If you really want to use MySQL, uncomment the following line
#image: mysql:8.0.27
command: '--default-authentication-plugin=mysql_native_password'
restart: always
secrets:
- db-password
volumes:
- db-data:/var/lib/mysql
networks:
- private
environment:
- MYSQL_DATABASE=example
- MYSQL_ROOT_PASSWORD_FILE=/run/secrets/db-password
frontend:
build:
context: frontend
target: development
ports:
- 3000:3000
volumes:
- ./frontend/src:/code/src
- /code/node_modules
networks:
- public
depends_on:
- backend
networks:
public:
private:
volumes:
back-notused:
db-data:
secrets:
db-password:
file: db/password.txt
include:
- template: "Verify/Accessibility.gitlab-ci.yml"
- template: "Security/SAST.gitlab-ci.yml"
- template: "Security/Dependency-Scanning.gitlab-ci.yml"
- template: "Security/Secret-Detection.gitlab-ci.yml"
stages:
- build
- test
- deploy
- accessibility
build-baldir:
stage: build
script:
- "echo do stuff"
deploy-baldir:
needs:
- "build-baldir"
script:
- "echo do stuff"
build-next:
stage: build
script:
- "echo do stuff"
deploy-next:
needs:
- "build-next"
script:
- "echo do stuff"
build-audit:
stage: accessibility
script:
- "echo do stuff"
deploy-audit:
stage: accessibility
script:
- "echo do stuff"
IDE can help you learning by proposing automatic fixes
Convert to Lambda expression
Convert to switch expression
Convert from markdown to asciidoc code listing 🥷
# Installation steps
1. Connect to machine A with SSH
2. Move to directory `/www`
3. Delete content
4. Download latest artifact
5. extract artifact in `/www`
6. verify new site is online
# Installation steps
## Prepare
1. Connect to machine A with SSH
## Cleanup content
```shell
rm -rf /www/*
```
## Install latest version
```shell
wget https://where-your-artifact-is/artifact.tar.gz
tar -xvf artifact.tar.gz
# cleanup
rm -f artifact.tar.gz
```
# Installation steps
1. Connect to machine A with SSH
2. Run preinstalled script `./full-install.sh`
3. Run healthcheck script `./healthcheck.sh`
Gitlab CI
Ansible
Terraform
Puppet
Solve one problem at a time
Your procedure will be reviewed by those who uses it and evolve
Lazy automation costs less
When/if you investigate for On the Shelf tools
you know what they need to solve
you may integrate your own scripts in the tools
Documentation is about Knowledge
That is worth sharing
Reliable
Low effort
Collaborative
Insightful
Your languages
Your tools and what they can produce
Generic knowledge
Favor conversations and working together
Knowledge is mostly already there
Where is it? Multiple places? Duplicated?
Is it up to date?
How many authors of the last updates have left the team?
Does documentation explains WHY or just HOW?
Can you trust the diagrams?
Is some knowledge redundant between the source and
I Built a 600 Meter Human Cannon That Ends All Existence - Satisfactory
Let’s Game It Out - 2020
It’s less dangerous to let developer experiment crazy stuff on doc instead of production code