Print the dependency tree for a given report using orderly log

Investigate the dependency structure in a set of orderly reports. This function allows the dependency graph to be created for set of reports that have been run and committed (the archive) or of a set of reports that could be run (the src) to be discovered and printed to screen. This is experimental and somewhat subject to change and improvement.

Usage

orderly_graph(
  name,
  id = "latest",
  root = NULL,
  locate = TRUE,
  direction = "downstream",
  propagate = TRUE,
  max_depth = Inf,
  recursion_limit = 100,
  show_all = FALSE,
  use = "archive"
)

Arguments

name: the name of the report
id: the id of the report, if omitted, use the id of the latest report
root: The path to an orderly root directory, or NULL (the default) to search for one from the current working directory if locate is TRUE.
locate: Logical, indicating if the configuration should be searched for. If TRUE and config is not given, then orderly looks in the working directory and up through its parents until it finds an orderly_config.yml file.
direction: A string indicating if we want to move up or down the tree permitted values are upstream, downstream
propagate: A boolean indicating if we want to propagate out of date through the tree
max_depth: A numeric, how far back should the tree go, this can be useful to truncate a very large tree. (default = Inf)
recursion_limit: A numeric, limit for depth of tree, if the tree goes beyond this then an error is thrown. (default = 100)
show_all: A boolean, should we show all reports in the tree, not just the latest.
use: Character string indicating what we read to infer the dependency tree. Current valid values are archive (the default), which reads from archive reports and src which reads from the source reports.

Value

An orderly tree object with the root corresponding to the given report.

Details

orderly allows a report to rely on the artefacts of one or more other orderly reports. This allows users to develop a network of interconnected reports where the output from report becomes the source of data for another. There are two natural questions that can develop around this workflow:

We have updated a report; what are the reports that depend on this so that we can re-run them?
We have a report that we want to re-run to ensure uses the latest information. Which other reports are used (directly or indirectly) by this report?

This function displays this information in an easily readable format. Allowing users to see the dependency tree and which reports are out of date and need to be re-run.

Remark

By default the tree is built using data from the local report database (see orderly_commit, orderly_db). This means that it will not find changes from a report that has not be run and committed. That is, if a user changes a report to use or create different artefacts this will not be picked up by the function until the reports are re-run and committed to the archive.

It is possible to generate a tree from the source reports by using use = "src" - this generates the "theoretical tree", and has no concept of being "up to date" or of ids.

Warning

This interface is considered experimental and may change without notice. Please do not depend on it in scripts as it may break things. Consider this a (hopefully) useful way of exploring the dependencies in your reports interactively - let us know what is missing and we'll try and build it out.

Examples

path <- orderly::orderly_example("demo")

id <- orderly::orderly_run("other", root = path, parameters=list(nmin=0))
#> [ name       ]  other
#> [ id         ]  20230621-105029-8045ed04
#> [ sources    ]  functions.R
#> [ parameter  ]  nmin: 0
#> [ start      ]  2023-06-21 10:50:29
#> [ data       ]  source => extract: 20 x 2
#> [ parameter  ]  nmin: 0
#> 
#> > extract$number <- extract$number + rnorm(1)
#> 
#> > write.csv(extract, "summary.csv", row.names = TRUE)
#> 
#> > png("graph.png")
#> 
#> > par(mar = c(15, 4, 0.5, 0.5))
#> 
#> > do_plot(extract)
#> 
#> > dev.off()
#> agg_png 
#>       2 
#> [ end        ]  2023-06-21 10:50:29
#> [ elapsed    ]  Ran report in 0.01828814 secs
#> [ artefact   ]  summary.csv: 3fac8347e152c84c96e6676413c718b7
#> [ ...        ]  graph.png: a70cdfd037035a0b34e71b921fc9de42
orderly::orderly_commit(id, root = path)
#> [ commit     ]  other/20230621-105029-8045ed04
#> [ copy       ]
#> [ import     ]  other:20230621-105029-8045ed04
#> [ success    ]  :)
#> [1] "/tmp/RtmpGRuIRx/file477a68fb72/archive/other/20230621-105029-8045ed04"
id <- orderly::orderly_run("use_dependency", root = path)
#> [ name       ]  use_dependency
#> [ id         ]  20230621-105029-c216e772
#> [ depends    ]  other@20230621-105029-8045ed04:summary.csv -> incoming.csv
#> [ start      ]  2023-06-21 10:50:29
#> 
#> > d <- read.csv("incoming.csv", stringsAsFactors = FALSE)
#> 
#> > png("graph.png")
#> 
#> > par(mar = c(15, 4, 0.5, 0.5))
#> 
#> > barplot(setNames(d$number, d$name), las = 2)
#> 
#> > dev.off()
#> agg_png 
#>       2 
#> 
#> > info <- orderly::orderly_run_info()
#> 
#> > saveRDS(info, "info.rds")
#> [ end        ]  2023-06-21 10:50:29
#> [ elapsed    ]  Ran report in 0.01266217 secs
#> [ artefact   ]  graph.png: a70cdfd037035a0b34e71b921fc9de42
#> [ ...        ]  info.rds: a95344a04f333a80618ceb17d6ce60de
orderly::orderly_commit(id, root = path)
#> [ commit     ]  use_dependency/20230621-105029-c216e772
#> [ copy       ]
#> [ import     ]  use_dependency:20230621-105029-c216e772
#> [ success    ]  :)
#> [1] "/tmp/RtmpGRuIRx/file477a68fb72/archive/use_dependency/20230621-105029-c216e772"
id <- orderly::orderly_run("use_dependency_2", root = path)
#> [ name       ]  use_dependency_2
#> [ id         ]  20230621-105029-ebe8b1a1
#> [ depends    ]  use_dependency@20230621-105029-c216e772:graph.png -> graph.png
#> [ start      ]  2023-06-21 10:50:29
#> 
#> > stopifnot(file.exists("graph.png"))
#> 
#> > saveRDS(orderly::orderly_run_info(), "third.rds")
#> [ end        ]  2023-06-21 10:50:29
#> [ elapsed    ]  Ran report in 0.002780914 secs
#> [ artefact   ]  third.rds: 917777b4bc790198dad59354e800f750
orderly::orderly_commit(id, root = path)
#> [ commit     ]  use_dependency_2/20230621-105029-ebe8b1a1
#> [ copy       ]
#> [ import     ]  use_dependency_2:20230621-105029-ebe8b1a1
#> [ success    ]  :)
#> [1] "/tmp/RtmpGRuIRx/file477a68fb72/archive/use_dependency_2/20230621-105029-ebe8b1a1"
orderly::orderly_graph("other", root = path)
#> other [20230621-105029-8045ed04]
#> └──use_dependency [20230621-105029-c216e772]
#>     └──use_dependency_2 [20230621-105029-ebe8b1a1]
#> 
orderly::orderly_graph("use_dependency_2", root = path,
                                 direction = "upstream")
#> use_dependency_2 [20230621-105029-ebe8b1a1]
#> └──use_dependency [20230621-105029-c216e772]
#>     └──other [20230621-105029-8045ed04]
#>