Download dependent reports

Download dependent reports from an orderly remote. This can only be used if the orderly_config.yml lists a remote. This allows for a centralised workflow where a central orderly store exists and holds the canonical copies of reports, from which versions can be downloaded into local stores.

Usage

orderly_pull_dependencies(
  name = NULL,
  root = NULL,
  locate = TRUE,
  remote = NULL,
  parameters = NULL,
  recursive = TRUE
)

orderly_pull_archive(
  name,
  id = "latest",
  root = NULL,
  locate = TRUE,
  remote = NULL,
  parameters = NULL,
  recursive = TRUE
)

orderly_push_archive(
  name,
  id = "latest",
  root = NULL,
  locate = TRUE,
  remote = NULL
)

Arguments

name: Name of the report to download dependencies for. Alternatively, the default of NULL is useful if you have already set the working directory to be the source directory.
root: The path to an orderly root directory, or NULL (the default) to search for one from the current working directory if locate is TRUE.
locate: Logical, indicating if the configuration should be searched for. If TRUE and config is not given, then orderly looks in the working directory and up through its parents until it finds an orderly_config.yml file.
remote: Description of the location. Typically this is a character string indicating a remote specified in the remotes block of your orderly_config.yml. It is also possible to pass in a directly created remote object (e.g., using orderly_remote_path(), or one provided by another package). If left NULL, then the default remote for this orderly repository is used - by default that is the first listed remote.
parameters: Parameters to pass through when doing dependency resolution. If you are using a query for id that involves a parameter (e.g., latest(parameter:x == p)) you will need to pass in the parameters here. Similarly, if you are pulling a report that uses query dependencies that reference parameters you need to pass them here (the same parameter set will be passed through to all dependencies).
recursive: Logical, indicating if all dependencies of a report should also be pulled. Setting this to FALSE only the direct reports, along with metadata for the dependencies; this will be potentially much faster, but leaves your archive in a more fragile state.
id: The identifier (for orderly_pull_archive). The default is to use the latest report.

Value

No return value, these functions are called only for their side effects

Details

The orderly_pull_archive function pulls report directly (without it being a dependent report).

After setting your username up you can run orderly_pull_dependencies("reportname") to pull the dependencies of "reportname" down so that "reportname" can be run, or you can run orderly_pull_archive("reportname") to pull a copy of "reportname" that has been run on the remote server.

Pulling an archive report from a remote also pulls its dependencies (recursively), and adds all of these to the local database. This may require migrating old orderly archives (orderly_migrate()). Note that this migration will likely fail for remote orderly versions older than 0.6.8 because the migration needs to read data files on disk that are not included in the downloaded archive in order to collect all the information required for the database. In this case, ask the administrator of the remote orderly archive to migrate their archive, and then re-pull.

Pushing an archive is possible only if the remote supports it. Currently this is supported by orderly_remote_path() remotes, though not by orderlyweb remotes. There is no control over what will accept a push at this point, nor any check that what you've pushed is "good" except that it exists in your archive. As with pulling an archive, pushes are recursive with respect to dependencies. The configuration interface here will likely change a little over time.

Examples

# Suppose we have a "remote" orderly repository at some path.
# This might be read-only for you in practice and available via a
# network filesystem or a dropbox folder synced to your computer.
# We'll populate this with a pair of reports:
path_remote <- orderly::orderly_example("demo")
id <- orderly::orderly_run("other", list(nmin = 0),
                           root = path_remote, echo = FALSE)
#> [ name       ]  other
#> [ id         ]  20230621-105033-eb061005
#> [ sources    ]  functions.R
#> [ parameter  ]  nmin: 0
#> [ start      ]  2023-06-21 10:50:33
#> [ data       ]  source => extract: 20 x 2
#> [ parameter  ]  nmin: 0
#> [ end        ]  2023-06-21 10:50:33
#> [ elapsed    ]  Ran report in 0.01761746 secs
#> [ artefact   ]  summary.csv: 3fac8347e152c84c96e6676413c718b7
#> [ ...        ]  graph.png: a70cdfd037035a0b34e71b921fc9de42
orderly::orderly_commit(id, root = path_remote)
#> [ commit     ]  other/20230621-105033-eb061005
#> [ copy       ]
#> [ import     ]  other:20230621-105033-eb061005
#> [ success    ]  :)
#> [1] "/tmp/RtmpGRuIRx/file47efd9e3e/archive/other/20230621-105033-eb061005"
id <- orderly::orderly_run("use_dependency",
                           root = path_remote, echo = FALSE)
#> [ name       ]  use_dependency
#> [ id         ]  20230621-105034-2e62a9d8
#> [ depends    ]  other@20230621-105033-eb061005:summary.csv -> incoming.csv
#> [ start      ]  2023-06-21 10:50:34
#> [ end        ]  2023-06-21 10:50:34
#> [ elapsed    ]  Ran report in 0.0123229 secs
#> [ artefact   ]  graph.png: a70cdfd037035a0b34e71b921fc9de42
#> [ ...        ]  info.rds: 5c05bfcb21a29c67577faa8bc6382927
orderly::orderly_commit(id, root = path_remote)
#> [ commit     ]  use_dependency/20230621-105034-2e62a9d8
#> [ copy       ]
#> [ import     ]  use_dependency:20230621-105034-2e62a9d8
#> [ success    ]  :)
#> [1] "/tmp/RtmpGRuIRx/file47efd9e3e/archive/use_dependency/20230621-105034-2e62a9d8"

# We'll create a an object to interact with this remote using
# orderly_remote_path.
remote <- orderly::orderly_remote_path(path_remote)

# We can use this object directly
remote$list_reports()
#>  [1] "changelog"          "connection"         "default-param"     
#>  [4] "global"             "html"               "interactive"       
#>  [7] "minimal"            "multi-artefact"     "multifile-artefact"
#> [10] "other"              "slow1"              "slow10"            
#> [13] "slow3"              "spaces"             "use_dependency"    
#> [16] "use_dependency_2"   "use_resource"       "use_resource_dir"  
#> [19] "view"              
remote$list_versions("other")
#> [1] "20230621-105033-eb061005"

# More typically one will interact with the functions
# orderly_pull_archive and orderly_pull_dependencies.

# Now, suppose that you have your "local" copy of this; it shares
# the same source (ordinarily these would both be under version
# control with git):
path_local <- orderly::orderly_example("demo")

# If we wanted to run the report "use_dependency" we need to have
# a copy of the report "other", on which it depends:
try(orderly::orderly_run("use_dependency", root = path_local))
#> [ name       ]  use_dependency
#> Error in orderly_find_report(id, name, config, draft = use_draft, must_work = TRUE) : 
#>   Did not find archive report other:latest

# We can "pull" dependencies of a report before running
orderly::orderly_pull_dependencies("use_dependency", remote = remote,
                                   root = path_local)
#> [ depends    ]  use_dependency has 1 dependency
#> [ pull       ]  other:20230621-105033-eb061005
#> [ import     ]  other:20230621-105033-eb061005

# Now we can run the report because we have a local copy of the
# dependency:
orderly::orderly_run("use_dependency", root = path_local)
#> [ name       ]  use_dependency
#> [ id         ]  20230621-105034-970fba55
#> [ depends    ]  other@20230621-105033-eb061005:summary.csv -> incoming.csv
#> [ start      ]  2023-06-21 10:50:34
#> 
#> > d <- read.csv("incoming.csv", stringsAsFactors = FALSE)
#> 
#> > png("graph.png")
#> 
#> > par(mar = c(15, 4, 0.5, 0.5))
#> 
#> > barplot(setNames(d$number, d$name), las = 2)
#> 
#> > dev.off()
#> agg_png 
#>       2 
#> 
#> > info <- orderly::orderly_run_info()
#> 
#> > saveRDS(info, "info.rds")
#> [ end        ]  2023-06-21 10:50:34
#> [ elapsed    ]  Ran report in 0.01296592 secs
#> [ artefact   ]  graph.png: a70cdfd037035a0b34e71b921fc9de42
#> [ ...        ]  info.rds: f40fcef7bb80452c0488305e6ab89058
#> [1] "20230621-105034-970fba55"

# We can also directly pull previously run reports:
orderly::orderly_pull_archive("use_dependency", id, remote = remote,
                              root = path_local)
#> [ pull       ]  use_dependency:20230621-105034-2e62a9d8
#> [ depends    ]  other/20230621-105033-eb061005
#> [ pull       ]  other:20230621-105033-eb061005 already exists, skipping
#> [ import     ]  use_dependency:20230621-105034-2e62a9d8
orderly::orderly_list_archive(root = path_local)
#>             name                       id
#> 1          other 20230621-105033-eb061005
#> 2 use_dependency 20230621-105034-2e62a9d8

Usage

Arguments

Value

Details

See also

Examples