forked from ThinkR-open/engineering-shiny-book
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathstructure.Rmd
726 lines (550 loc) · 28.6 KB
/
structure.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
# Structuring your Project {#structure}
## Shiny App as a Package
In the next chapter you'll be introduced to the `{golem}` package, which is __an opinionated framework for building production-ready Shiny Applications__.
This framework starts by creating a package skeleton waiting to be filled.
But, in a world where Shiny Applications are mostly created as a series of files, why bother with a package?
### What's in a Shiny App?
OK, so let's ask the question the other way round.
Think about your last Shiny which was created as a single-file (`app.R`) or two files app (`ui.R` and `server.R`).
You've got these two, and you put them into a folder.
So, let's have a review of __what you'll need next for a robust application__.
First, __metadata.__
In other words, the name of the app, the version number (which is crucial to any serious, production-level project), what the application does, who to contact if something goes wrong.
Then, you need to find a way to __handle the dependencies__.
Because you know, when you want to push your app into production, you can't have this conversation with IT:
>IT: Hey, I tried to 'source("app.R")' but I've got an error.
>
>R-dev: What's the error?
>
>IT: It says "could not find package 'shiny'".
>
>R-dev: Ah yes, you need to install {shiny}. Try to run 'install.packages("shiny")'.
>
>IT: OK nice. What else?
>
>R-dev: Let me think, try also 'install.packages("DT")'... good? Now try 'install.packages("ggplot2")', and ...
>
>[...]
>
>IT: Ok, now I source the 'app.R', right?
>
>R-dev: Sure!
>
>IT: Ok so it says 'could not find function runApp()'
>
>R-dev: Ah, you've got to do library(shiny) at the beginning of your script. And library(purrr), and library(jsonlite)*.
`*` Which will lead to a Namespace conflict on the `flatten()` function that can cause you some debugging headache.
So, hey, it would be cool if we could have a Shiny app that only imports specific functions from a package, right?
__So yes, dependencies matter. You need to handle them, and handle them correctly. __
Now, let's say you're building a big app.
Something with thousands of lines of code.
Handling a one-file or two-file shiny app with that much lines is just a nightmare.
So, what to do?
Let's split everything into smaller files that we can call!
And finally, we want our app to live long and prosper, which means we need to document it: __each small pieces of code should have a piece of comment__ to explain what these specific lines do.
The other thing we need for our application to be successful on the long term is tests, so that we are sure we're not introducing any regression.
Oh, and that would be nice if people can get a `tar.gz` and install it on their computer and have access to a local copy of the app!
OK, so let's sum up: we want to build an app.
This app needs to have __metadata__ and to handle __dependencies__ correctly, which is what you get from the `DESCRIPTION` + `NAMESPACE` files of the package.
Even more practical is the fact that you can do "selective namespace extraction" inside a package, i.e you can say "I want this function from this package".
Also, __this app needs to be split up in smaller `.R` files__, which is the way a package is organized.
And I don't need to emphasize how __documentation__ is a vital part of any package, so we solved this question too here. So is the __testing toolkit__.
And of course, the "install everywhere" wish comes to life when a Shiny App is in a package.
### The other plus side of Shiny as a Package
#### Testing
__Nothing should go to production without being tested. Nothing.__
Testing production apps is a wide question, and I'll just stick to tests inside a Package here.
Frameworks for package testing are robust and widely documented.
So you don't have to put any extra-effort here: just use a canonical testing framework like [`{testthat}`](https://testthat.r-lib.org/).
Learning how to use it is not the subject of this chapter, so feel free to refer to the documentation.
See also Chapter 5 of ["Building a package that lasts"](https://speakerdeck.com/colinfay/building-a-package-that-lasts-erum-2018-workshop?slide=107).
What should you test?
+ First of all, as we've said before, the app should be split between the UI part and the back-end (or 'business logic') part.
These back-end functions are supposed to run without any interactive context, just as plain old functions. So for these ones, __you can do classical tests__.
As they are back-end functions (so specific to a project), `{golem}` can't provide any helpers for that.
+ For the UI part, __remember that any UI function is designed to render an HTML element__.
So you can save a file as HTML, and then compare it to a UI object with the `golem::expect_html_equal()`.
```{r structure-1, eval = FALSE}
library(shiny)
ui <- tagList(h1("Hello world!"))
htmltools::save_html(ui, "ui.html")
golem::expect_html_equal(ui, "ui.html")
# Changes
ui <- tagList(h2("Hello world!"))
golem::expect_html_equal(ui, "ui.html")
```
This can for example be useful if you need to test a module.
A UI module function returns an HTML tag list, so once your modules are set you can save them and use them inside tests.
```{r structure-2, eval = FALSE}
my_mod_ui <- function(id){
ns <- NS("id")
tagList(
selectInput(ns("this"), "that", choices = LETTERS[1:4])
)
}
my_mod_ui_test <- tempfile(fileext = "html")
htmltools::save_html(my_mod_ui("test"), my_mod_ui_test)
# Some time later, and of course saved in the test folder,
# not as a temp file
golem::expect_html_equal(my_mod_ui("test"), my_mod_ui_test)
```
`{golem}` also provides two functions, `expect_shinytag()` and `expect_shinytaglist()`, that test if an object is of class `"shiny.tag"` or `"shiny.tag.list"`.
+ Testing package launch: when launching `golem::use_recommended_tests()`, you'll find a test built on top of `{processx}` that allows to check if the application is launch-able. Here's a short description of what happens:
```{r structure-3, eval = FALSE}
# Standard testthat things
context("launch")
library(processx)
testthat::test_that(
"app launches",{
# We're creating a new process that runs the app
x <- process$new(
"R",
c(
"-e",
# As we are in the tests/testthat dir, we're moving
# two steps back before launching the whole package
# and we try to launch the app
"setwd('../../'); pkgload::load_all();run_app()"
)
)
# We leave some time for the app to launch
# Configure this according to your need
Sys.sleep(5)
# We check that the app is alive
expect_true(x$is_alive())
# We kill it
x$kill()
}
)
```
_Note_: this specific configuration will possibly fail on Continuous integration platform as Gitlab CI or Travis. A workaround is to, inside your CI yml, first install the package with `remotes::install_local()`, and then replace the `setwd (...) run_app()` command with `myuberapp::run_app()`.
For example:
- in `.gitlab-ci.yml`:
```
test:
stage: test
script:
- echo "Running tests"
- R -e 'remotes::install_local()'
- R -e 'devtools::check()'
```
- in `test-golem.R`:
```{r structure-4, eval = FALSE}
testthat::test_that(
"app launches",{
x <- process$new(
"R",
c(
"-e",
"datuberapp::run_app()"
)
)
Sys.sleep(5)
expect_true(x$is_alive())
x$kill()
}
)
```
#### Documenting
Documenting packages is a natural thing for any R developer.
Any exported function should have its own documentation, hence you are "forced" to document any user facing-function.
Also, building a Shiny App as a package allows you to write standard R documentation:
- A `README` at the root of your package
- `Vignettes` that explain how to use your app
- A `{pkgdown}` that can be used as an external link for your application.
### Deploy
#### Local deployment
As your Shiny App is a standard package, it can be built as a `tar.gz`, sent to your colleagues, friends, and family, and even to the CRAN.
It can also be installed in any R-package repository. Then, if you've built your app with `{golem}`, you'll just have to do:
```{r structure-5, eval=FALSE}
library(myuberapp)
run_app()
```
to launch your app.
#### RStudio Connect & Shiny Server
Both these platforms expect a file app configuration, i.e an `app.R` file or `ui.R` / `server.R` files.
So how can we integrate this "Shiny App as Package" into Connect or Shiny Server?
+ Using an internal package manager like [RStudio Package Manager](https://www.rstudio.com/products/package-manager/), where the package app is installed, and then you simply have to create an `app.R` with the small piece of code from the section just before.
+ Uploading the package folder to the server. In that scenario, you use the package folder as the app package, and upload the whole thing. Then, write an `app.R` that does:
```{r structure-6, eval = FALSE}
pkgload::load_all()
shiny::shinyApp(ui = app_ui(), server = app_server)
```
And of course, don't forget to add this file in the `.Rbuildignore`!
This is the file you'll get if you run `golem::add_rconnect_file()`.
#### Docker containers
In order to dockerize your app, simply install the package as any other package, and use as a `CMD` `R -e 'options("shiny.port"=80,shiny.host="0.0.0.0");myuberapp::run_app()'`.
Of course changing the port to the one you need.
You'll get the Dockerfile you need with `golem::add_dockerfile()`.
### Resources
+ [R packages](http://r-pkgs.had.co.nz/)
+ ["Building a package that lasts"](https://speakerdeck.com/colinfay/building-a-package-that-lasts-erum-2018-workshop)
+ [Writing R Extensions](https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Creating-R-packages)
+ [R package primer - a minimal tutorial](https://kbroman.org/pkg_primer/)
## Using Shiny Modules
### Small is beautiful
Modules are one of the most powerful tool for building Shiny Application.
But what are they?
> Shiny modules address the namespacing problem in Shiny UI and server logic, adding a level of abstraction beyond functions
`r right_link("Modularizing Shiny app code", "https://shiny.rstudio.com/articles/modules.html")`
Let's first untangle this quote with a simple example about what is shiny namespace problem.
#### One million "Validate" buttons
Shiny needs its outputs and inputs to have a __unique id__.
And unfortunately, we can't bypass that: when you send a plot __from R to the browser__, i.e from the `server` to the `ui`, the browser needs to know exactly where to put this element.
This "exactly where" is handled through the use of an `id`.
Ids are not a Shiny specific concept: they are at the very root of the way web pages work.
Understanding all of this is not the purpose of this chapter: just remember that Shiny inputs and outputs ids __have__ to be unique, so that the browser knows where to put what it receives from R, and R knows what to listen to from the browser.
The need to be unique is made a little bit complex by the way Shiny handles the names, as it share a global pool for all the id names, with no native way to use namespaces.
Namespaces are a computer science concept which has been designed to handle a common issue: how to share the same name for a variable in various place of your program without them conflicting.
In other words, how to use an object called `foo` several times in the program, and still be sure that it's correctly used depending on the context.
R itself has a system for namespaces ; that's what packages do and why you can have `purrr::flatten` and `jsonlite::flatten`: the function name is the same, but the two live in different namespaces, and they can mean two different things, as the symbol is evaluated inside two different namespaces.
If you want to learn more about namespaces, please refer to the [7.4 Special environments](https://adv-r.hadley.nz/environments.html#special-environments) chapter from _Advanced R_.
So, that's what modules are made for: creating small namespaces where you can safely define `ids` without conflicting with other ids in the app.
Why do we need to do that?
Think about the number of times you've created a "ok" or "validate" button.
How do you handle that?
By creating `validate1`, `validate2`, and so on and so forth.
And if you think about it, you're mimicking a namespacing process: a `validate` in namespace `1`, another in namespace `2`.
Consider this piece of code:
```{r structure-7, eval = FALSE}
library(shiny)
ui <- function(request){
fluidPage(
sliderInput("choice1", "choice 1", 1, 10, 5),
actionButton("validate1", "Validate choice 1"),
sliderInput("choice2", "choice 2", 1, 10, 5),
actionButton("validate2", "Validate choice 2")
)
}
server <- function(
input,
output,
session
){
observeEvent( input$validate1 , {
print(input$choice1)
})
observeEvent( input$validate2 , {
print(input$choice2)
})
}
shinyApp(ui, server)
```
This, of course, is an approach that works.
Well, it works as long as your code base is small.
But how can you be sure that you're not creating `validate6` on line 55 and another on line 837?
How can you be make sure that you're deleting the correct combination of UI/server components if they are named that way?
Also, how do you work smoothly in a context where you have to scroll from `sliderInput("choice1"` to `observeEvent( input$choice1 , {` which might be separated by thousands of lines?
#### A bite-sized code base
And of course, you know the saying that _"if you copy and paste something more than twice, you should make a function"_, so how do we refactor this piece of code so that it's reusable?
Yes, you guessed right: using shiny modules.
Shiny modules aim at three things: simplifying id namespacing, split the code base into a series of functions, and allow UI/Server parts of your app to be reused.
Most of the time, modules are used to do the two first: I'd say that 90% of the module I write are never reused^[
Most of the time, pieces / panels of the app are to unique too be reused elsewhere.
] ; they are here to allow me to split the code base into smaller, more manageable pieces.
With Shiny modules, you'll be writing a combination of UI and server functions.
Think of them as small, standalone Shiny apps, which output and handle a fraction of your global application.
If you've been developing R packages, you'd probably trying to split your functions into series of smaller functions, that's the exact same thing: you are, with just a little bit of tweaking, doing the same thing.
That is to say creating smaller functions that are easier to understand, develop and maintain.
### A practical walk through
#### Your first Shiny Module
So, here is how you'd refactor the example from before with modules:
```{r structure-8, eval = FALSE}
name_ui <- function(id){
ns <- NS(id)
tagList(
sliderInput(ns("choice"), "Choice", 1, 10, 5),
actionButton(ns("validate"), "Validate Choice")
)
}
name_server <- function(input, output, session){
ns <- session$ns
observeEvent( input$validate , {
print(input$choice)
})
}
library(shiny)
ui <- function(request){
fluidPage(
name_ui("name_ui_1"),
name_ui("name_ui_2")
)
}
server <- function(
input,
output,
session
){
callModule(name_server, "name_ui_1")
callModule(name_server, "name_ui_2")
}
shinyApp(ui, server)
```
Let's stop for a minute and decompose what we've got here.
The __server__ function is pretty much the same as before: you'll just be using the same code as the one you've been using so far.
The __ui__ function has some new things in it. Well, two new things, which are `ns <- NS(id)` and `ns(inputId)`.
That's where the namespacing happens.
You can think about this function as a way to add a namespace to your id: you've been doing `validate1` and `validate2` before, now you're doing this with the function created by `ns <- NS(id)`.
You'll find this little piece of code on top of all the module ui functions.
To understand what it does, let's try and run it outside of Shiny:
```{r structure-9}
id <- "name_ui_1"
ns <- NS(id)
ns("choice")
```
And here it is, our namespaced id!
And of course, calling it with various id will create various namespaces for the id, preventing you from id conflicts^[
Well, of course you can still have inner module id conflicts, but they are easier to avoid, detect, and fix.
].
All you have to do now is to make sure that ids are unique at the "upper" levels.
Then you can have as many `validate` input as you want in your app: as long as this `validate` is unique inside your module you're good to go.
The `app_ui` contains a series of call to `module_ui_function(unique_id, ...)` with potential parameters:
```{r structure-10, eval = FALSE}
name_ui <- function(id, butname){
ns <- NS(id)
tagList(
actionButton(ns("validate"), butname)
)
}
name_ui("name_ui_1", "Validate Choice")
name_ui("name_ui_2", "Validate Choice, again")
```
```
<button id="name_ui_1-validate" type="button" class="btn btn-default action-button">Validate Choice</button>
<button id="name_ui_2-validate" type="button" class="btn btn-default action-button">Validate Choice, again</button>
```
The `app_server` side contains a series of `callModule(module_server_function, unique_id, ...)`, with potential parameters.
#### Passing args to your modules
Shiny modules will potentially be reused.
It's not the general pattern, but they can.
In that case, you'll potentially be using extra arguments to generate the UI and server conditionally.
Let's for example have a look at [mod_dataviz.R](https://github.com/ColinFay/tidytuesday201942/blob/master/R/mod_dataviz.R#L17) from the `{tidytuesday201942}` Shiny application.
This application contains 6 tabs, 4 of them being pretty much alike: a side bar with inputs, an a main panel with a button and the plot.
This is a typical case where you should reuse modules: if two or more parts are relatively similar, it's easier to bundle it inside a reusable module, and condition the ui/server with function arguments.
![](img/tidytuesdayapp.png)
Here, are some examples of how it works in the UI:
```{r structure-11, eval = FALSE}
mod_dataviz_ui <- function(id, type = c("point", "hist", "boxplot", "bar")){
h4(
sprintf( "Create a geom_%s", type )
),
if (type == "boxplot" | type =="bar") {
selectInput(
ns("x"),
"x",
choices = names_that_are(c("logical", "character"))
)
} else {
selectInput(
ns("x"),
"x",
choices = names_that_are("numeric")
)
}
}
```
And in the server:
```{r structure-12, eval = FALSE}
mod_dataviz_server <- function(input, output, session, type){
if (type == "point"){
x <- rlang::sym(input$x)
y <- rlang::sym(input$y)
color <- rlang::sym(input$color)
r$plot <- ggplot(
big_epa_cars,
aes(!!x, !!y, color = !!color)
) +
geom_point() +
scale_color_manual(
values = color_values(
1:length(unique(pull(big_epa_cars, !!color))),
palette = input$palette
)
)
}
}
```
Then, the app server is:
```{r structure-13}
app_server <- function(input, output,session) {
#callModule(mod_raw_server, "raw_ui_1")
callModule(mod_dataviz_server, "dataviz_ui_1", type = "point")
callModule(mod_dataviz_server, "dataviz_ui_2", type = "hist")
callModule(mod_dataviz_server, "dataviz_ui_3", type = "boxplot")
callModule(mod_dataviz_server, "dataviz_ui_4", type = "bar")
}
```
And the UI:
```{r structure-14}
app_ui <- function() {
# [...]
tagList(
fluidRow(
id = "geom_point", mod_dataviz_ui("dataviz_ui_1", "point")
),
fluidRow(
id = "geom_hist", mod_dataviz_ui("dataviz_ui_2", "hist")
)
)
}
```
### Communication between modules
One of the hardest part about modules is sharing data across them.
There are at least two approaches: returning reactive, or the "stratégie du petit r" (to be pronounced with a french accent).
#### Returning values from the module
One common approach is return a reactive from one module, and to pass it to another.
```{r structure-15, eval = FALSE}
name_ui <- function(id){
ns <- NS(id)
tagList(
sliderInput(ns("choice"), "Choice", 1, 10, 5)
)
}
name_server <- function(input, output, session){
ns <- session$ns
return(
reactive({
input$choice
})
)
}
name_b_ui <- function(id){
ns <- NS(id)
tagList(
actionButton(ns("validate"), "Print")
)
}
name_b_server <- function(input, output, session, react){
ns <- session$ns
observeEvent( input$validate , {
print(react())
})
}
library(shiny)
ui <- function(request){
fluidPage(
name_ui("name_ui_1"),
name_b_ui("name_ui_2")
)
}
server <- function(
input,
output,
session
){
res <- callModule(name_server, "name_ui_1")
callModule(name_b_server, "name_ui_2", react = res)
}
shinyApp(ui, server)
```
That works well, but for large Shiny Apps it might be hard to handle large list of reactive outputs / inputs.
It might also create some reactivity issues, as they are harder to control.
#### The "stratégie du petit r"
With this strategy, instead of passing reactives as function input, we'll be creating a global reactive list which is passed along other modules.
The idea is that it allows us to be less preoccupied about what your module takes as input.
Here, we will be creating a "global" `reactiveValues()` that we will pass downstream.
```{r structure-16, eval = FALSE}
name_ui <- function(id){
ns <- NS(id)
tagList(
sliderInput(ns("choice"), "Choice", 1, 10, 5)
)
}
name_server <- function(input, output, session, r){
ns <- session$ns
observeEvent( input$choice , {
r$choice <- input$choice
})
}
name_b_ui <- function(id){
ns <- NS(id)
tagList(
actionButton(ns("validate"), "Print")
)
}
name_b_server <- function(input, output, session, r){
ns <- session$ns
observeEvent( input$validate , {
print(r$choice)
})
}
library(shiny)
ui <- function(request){
fluidPage(
name_ui("name_ui_1"),
name_b_ui("name_ui_2")
)
}
server <- function(
input,
output,
session
){
r <- reactiveValues()
callModule(name_server, "name_ui_1", r)
callModule(name_b_server, "name_ui_2", r)
}
shinyApp(ui, server)
```
The plus side of this method is that whenever you're add something in one module, it's immediately available in the other modules.
The down side is that it can be harder to reason about the app, as the input / content of the `r` is not specified anywhere: as you don't pass any arguments to your function other than `r`, you don't really know what's inside.
Also, not that if you want to share your module, for example in a package, you should document the structure of the `r`.
For example:
```
#' @param r a `reactiveValues()` with a `choice` element in it. This `r$choice` will be printed to the R console.
```
#### A third way
There is another way to share data across modules, which is creating an R6 object which is then pass along the modules.
In the spirit, it's more or less the same as passing the `r` list, except that it is not a reactive object, making it more robust to the complexity of handling reactivity invalidation across modules.
This methods is explain in the chapter [Reactivity anti-patterns](#optim-caveat) of this book.
#### When should you modularize?
From the very beginning.
The overhead of writing a module compared to putting everything inside the app function is relatively low: it's even simpler if you are working in a framework like `{golem}`, which promotes the use of modules from the very beginning of your application.
> "Yes but I just want to write a small app, nothing fancy
Production apps almost always started as a small POC.
Then the small POC becomes an interesting idea.
Then this idea becomes a strategical asset.
And before you know it your 'not-that-fancy' app needs to become larger
## Splitting your app into files
### Small is beautiful (bis repetita)
There is nothing harder to maintain than a Shiny app which is only made of one 1000 lines long `app.R`.
Well, there still is the 10 000 lines long `app.R`, but you've got the idea.
Long scripts are almost always synonym of complexity when it comes to building an application.
Of course, small and numerous scripts don't systematically prevent from codebase complexity but they simplify collaboration and maintenance, and of course divide the application logic into smaller, easier to understand bits of code.
So yes, big files are complex to handle and make development harder.
Indeed, here is what happens when you're working on a production application:
+ You'll be working during a long period of time (either in one run or split across several months) on your codebase, meaning that you'll have to get back to pieces of code you have written a long time ago.
+ You'll possibly be developing with others.
Maintaining a code base when several persons are working on the files is already a complex thing: from time to time you might be working on the same file separately, a situation where you'll have to be careful about what and how you merge things when changes are implemented.
Of course, it's almost impossible to work on one same file all along the project without losing your mind: even more if this file is thousands of lines long.
+ You'll be implementing numerous features.
Numerous features imply a lot of UI & server interaction.
And in an `app.R` file of thousands of line, it's very hard to match the UI element with its server counterpart: when the UI is on line 50 and the server on line 570, you'll be scrolling a lot when working on that element.
So yes, there are a lot of reasons for splitting your application into smaller pieces: it's easier to maintain, easier to decipher, and of course it facilitates collaboration.
### Conventions matter
So, now that we've talk about the benefits of splitting files, let's think about how to do that.
Splitting files is good, splitting files using a defined convention is better.
Why?
Because using a common convention for your files helps the other developers (and potentially you) to know exactly what is contained in a specific file, and that way it helps everybody know where to look for debugging / implementing new features.
For example, if you follow `{golem}`'s convention, you'll know that a file starting with `mod_` contains a module, so if I take over a project, look in the `R/` folder and see files starting with these three letters, I'll know immediately what these files contain.
That's what we'll see in this part: a proposition for a convention on how to split your application into smaller pieces.
First of all, put everything into an `R/` folder.
If you're building your app using the `{golem}` framework, this is already what you're doing: using this package convention to hold the functions for your application.
Once you've got this, here is the `{golem}`-specific convention for organizing your files:
+ `app_*.R` (typically `app_ui.R` and `app_server.R`) should contain the top level functions which are used to defined your user interface and your server function.
+ `fct_*` are files that contains business logic, potentially large functions.
They are the backbone of the app but are potentially not specific to a given module.
They can be added in `{golem}` with the `add_fct("name")` function.
+ `mod_*` are files that contain ONE module.
As many shiny apps contains a series of tabs, or at least a tab-like pattern, we suggest that you number then according to their step in the application.
And as tabs are almost always named, you can use the tab-name as the file name.
For example, if you're building a dashboard and the first tab is called "Import", your should name your file `mod_01_import.R`, which you can create with `golem::add_module("01_import")`.
Note that when building a module file with `{golem}`, you can add a `fct_` and `utils_` specific file, that will hold functions and utilities for this specific modules.
For example, `golem::add_module("01_import", fct = "readr", utils = "ui")` will create `R/mod_01_import.R`, `R/mod_01_import_fct_readr.R` and `R/mod_01_import_utils_ui.R`.
+ `utils_*` are files that contain utilities, which are small helper functions.
For example, you might want to have a `not_na`, which is `not_na <- Negate(is.na)`, a `not_null`, or small tools that you'll be using application-wide.
Note that you can also create `utils` for a specific module.
+ `*_ui_*`, for example `utils_ui.R`, relates to the user interface. Anything back-end can be noted using `*_server_*`, for example `fct_connection_server.R` will contain functions that are related to the connection to a database, and which are specifically used from the server side.
Of course, as with any convention, you might be deviating from time to time from this pattern.
Your app may not have that many functions, or maybe the functions can all fit into one `utils_` file.
But be it one or thousands of file, it's always a good practice to stick to a formalized pattern.