An Introduction to Publishing Data Products Using

McCrea Cobb

Refuge Inventory and Monitoring
mccrea_cobb@fws.gov
mccrea-cobb

https://github.com/USFWS/posit-connect-presentation

• Hello everybody. My name is McCrea Cobb. I work as a biometrician in the Alaska Refuge Inventory and Monitoring Program.
• I’m here to talk to you about an exciting and (fairly) new tool that is available to everyone called Posit Connect.

• Those of you that have moved to a script-based data workflow can speak to the advantages of this approach over a traditional manual data workflow.
• Advantages include reproducibility, transparent and documented steps, and virtually unlimited tools for analyzing and visualizing data.
• [click]
• One of the great features of programming languages like R and Python is the ability to generate products from data.
• These products include basic tables and plots, but more recently, they also include the ability to render dynamic reports that update with new data or interactive applications where a user can select different options to visualize and summarize data.
• [click]
• If you using code for your work, you might have heard about or created reports rendered in RMarkdown or Quarto or created applications using R Shiny.
• The possibilities for communicating data using code can seem endless if you have the time and know-how.

• But sharing these reports and applications can be a limitation.
• For example, say you have a report that you wrote in Quarto. How do you share that will the decision-maker?
• Most likely, the answer might be that you render the report as a PDF or Word document, save it on your computer (or better yet ServCat!), and then email it as an attachment or a hyperlink to the ServCat reference.
• What about applications? You’ve developed a cool Shiny app to display your survey results, but how do you share that with managers that aren’t R users? [click]
• Until fairly recently, FWS biologists did not have any viable options for publishing R Shiny applications.

• This was quite frustrating, given the time and energy put into generating these cool data products.

• Enter Posit Connect!
• Posit Connect is a publishing platform that supports R and Python frameworks, including Shiny, Quarto, RMarkdown, and Jupyter Notebooks.
• It allows you to:
  • Publish applications (Shiny) and documents (Quarto/R Markdown)
  • Automate and schedule tasks with code
  • Host resources like data, models, and R objects
  • Even develop and share web APIs

What is Quarto ?

Illustration by Allison Horst

• Before diving another further, I realize that there are many folks listening that may not be familiar with some of the systems that I’ve mentioned.
• So, here’s a quick overview.
• Quarto is an open source scientific publishing system
• Next generation of R Markdown
• Supports multiple programming languages (R, Python, Julia, Observable)
• Can create articles, reports, presentations, websites
• Mix text with in-line code to update statistics in the text
• Can output in HTML, PDF, Word and other formats

What is R Shiny ?

• Shiny is an R package
• Enables building interactive web applications that execute R
  • Standalone websites
  • Embed interactive charts in Quarto docs
  • Build dashboards
• Primary advantage of a Shiny app is that decision makers can interact with the data.
• The Shiny app example show here uses data from Alaska refuge survey annual reporting to explore refuge survey progress along a data life cycle from data acquisition to preservation, and some of the reasons for why annual survey tasks could not be completed.
• For example, in 2023, a lack of refuge staff was the primary reason that surveys that did not acquire data, followed by partner resources.

Publishing a Report

Ok. I’ll now walk through the steps to publishing a Quarto report.

Quarto file

Rendered Report

• On the left side, we have a Quarto markdown file to render a simulated moose aerial survey. The code imports observation data, creates figures and tables, calculates statistics that are included in the text.
  
• Typical of these files, there is what’s called a YAML header at the top that provides from metadata and the file format to output (in this case, it is HTML).
• [click]
• When rendered in Rstudio, this code produces a HTML report. It includes an embedded and interactive leaflet map widget and interactive tables of the data. In line summary statistics are calculated based on the input data.

• Ok, here is the same Quarto file on the left and the rendered report on the right, as viewed in RStudio. I’ll walk through the steps to get this local report published as a website in Posit Connect.

Scheduling a Report

• Update report in reponse to new data:
  1. Include code to import data in the qmd
  2. Create a second qmd to import data, save as a pin
• Option to email report on a schedule

• What if you have a report that you regularly update or one that needs to be shared with decision-makers on a schedule?
• In this sitation, you’re looking at scheduling a report, which is quite straightforward in Posit Connect.
• The first steps are the same as what you just saw.
• When you get to the window asking whether to publish the doc only or publish it with the source code, you select the later option.
• Note that to have the report update with new data, you need to provide the Quarto doc with access to updated data. This can be done either within the same Quarto file, by importing data that are updated (for example from AGOL) or you can create a seperate Quarto file that would be scheduled to import data and save it in Posit Connect as what’s called a Pin. You would then point to this Pin in your original Quarto report.
• Either way, you have options for how often to update the data or report. You can also tell it to schedule an email with the updated report, so that decision makers know that new information is available.

Publishing a Application

Almost the same process
as publishing a report!

The steps are basically the same as publishing a Quarto report.

Publishing Git-backed Content

1. Create a Version Control RStudio Project connected to a GitHub repository.
2. Create a manifest.json file by running rsconnect::writeManifest().
3. Visit Posit Connect. Select .
4. Select .
5. Provide URL to public GitHub repository. Select a branch.
6. 🤩

• For those of you that are using Git and GitHub to version control and collaborate on developing data prodcuts, know that it’s possible to publish content on Posit Connect that updates with changes on a GitHub repository.
• Why would you want this workflow?
  • Automatically updates application when changes are pushed to GitHub
  • Can dedicate GitHub branches for your app in either a development and production environment
• First, you need to create project that is under version control and connected to a GitHub repository.
• Then you need to create what’s called a manifest json file that describes the R package dependencies and environment. To do this, simply run the function shown.
• Next, you need to open your Posit Connect content dashboard and select “Publish”.
• Select the Import from Git option.
• Provide a URL to a public GitHub repository and specify a branch.
• That’s it.
• Whenever you make changes to that branch, Posit Connect detects that change and republishes your data product.

Getting Started

Annual Costs

License: $290
Environment: $144
Total per app: $434 + $52 initial setup fee

Contact

Michaelle Schultz
Chief of Data Integration
FWS Ecological Services
michaelle_schultz@fws.gov

Additional Resources

https://github.com/USFWS/posit-connect-presentation

ServCat Reference: https://ecos.fws.gov/ServCat/Reference/Profile/172724