Part 3 in a multi part series on Azure DevOps pipelines

Miss a post take a look back at part 1 and part 2

Welcome to the next post in the Azure DevOps Multi-stage Pipelines series! In the last two posts we built a full pipeline in yaml. For our little project, this is a complete pipeline and it could just be called done here. However, there are always things to improve and learn. In this post I’ll introduce a couple concepts that can be used to enhance the pipeline and help with the security, simplicity and reusability.

At the end of this post we will have some variable groups set up to prevent from checking in secure information into source control. We will also utilize templates to reduce the number of lines in the base pipeline and allow for reusability of sections of the code.

Requirements to follow along

  • Azure Subscription – Sign up for a free account
  • Azure DevOps Account – Sign up for a free account
  • Repository – Any Git repository can be used and connected to Azure Pipelines but this walkthrough will utilize Azure Repos Git repository
  • IDE – This walkthrough was created using Visual Studio Code which has extensions for Pipeline syntax highlighting

Base project

We will be continuing with the .Net Core API project and pipeline that have been used through the series. You can follow along in the first two posts and then pick up from there or grab the code from the branch ‘post2-release’ as a starting point for this post (https://github.com/cashewshideout/blog-azurepipeline/tree/post2-release).  It is not necessary to have previous knowledge of .Net Core for this walkthrough; the concepts of creating the Pipeline are universal between all supported languages.

Series Outline

Check out all of the posts in this series.

  1. Intro and setting up the build steps
  2. Deployment steps, environments and approvals
  3. Pipeline templates and variables (this post)

Variable Groups

All the way back in the first post in the series I did a quick introduction to variables. These are similar to variables you see elsewhere – aliases for a piece of data that will be used in multiple places. When this data needs to be changed then it will only need to be update in one place.

The in-code variables are very useful and should be utilized, however they may not be appropriate in all scenarios. What about times where the variable data needs to change depending on the environment being deployed to, or if this data is something that should not be kept in source control like a password or secret key?

If this data needs to change based on the environment then it is certainly possible to create a variable per environment but then the code will have un-needed repetition (eg. variables named devSetting and prodSetting).

A solution to this is to use variable groups available in Azure DevOps. Variable groups allow you to create a collection of variables that can be used in your pipeline. A group can be created for each environment or some other grouping and can even be shared across multiple pipelines.

The pipeline we are working with doesn’t have any specific places where this will be needed but let’s create a couple variable groups so we can see how they work.

In Azure DevOps under Pipelines, select Library to access where to create Variable groups. Select the button in the middle (+ Variable group) to create our first group.

On the Variable group creation page there are a variety of things to set.

First, we always want to make sure that they are appropriately named and given a detailed description.

The first toggle is to allow access to this variable group to all pipelines. Keep this toggled on for now, but in the resources section I have a link that describes how to only allow access for a single pipeline.

The next toggle is to allow for linking secrets from Azure Key Vault. You will be asked to connect to a subscription and enter a Key Vault name. Right now that is out of scope for this post so we will leave it toggled off.

The last thing to add are the actual variable Names and Values. Keep the names short but descriptive and remember that these names could be used across multiple environments so don’t get too specific.

It’s a secret

One last thing to note is the second variable here, named Secret, I actually don’t want the value to show in the logs or for others to see it on the screen. To achieve this we will click the lock to the right of the row.

Before saving, it changes the value to a collection of asterisks and you can toggle between this and plain text. However, after saving with the variable set as a secret you will no longer be able to view what the value was.

After changing the Secret variable to a secret, click Save at the top.

Production Variables

We want to have a variable group with the same variables for production. Luckily there is a Clone button next to the Save button. Click this and update the name, description and Readme variables.

Notice that the Secret variable has an error – variables marked as secret are not automatically copied. The Value for the variable named Secret will need to be re-entered.

You should now have two Variable groups

Pipeline Integration

Now that we have these variables in Azure DevOps, lets utilize them in our pipeline. Two additions have been made to this snippet of the staging deploy section.

variables (line 7 – 8): The variables property, similar to the global variables property towards the top of the pipeline will allow you to specify key/value variables. The variables in this section will be scoped only to this stage. There is a special option for this property – group where we can specify the name of the Variable group we created above. script (line 21 – 23): Like I mentioned previously this pipeline doesn’t meet the scenarios I mentioned above to utilize Variable groups. To demonstrate the concept though, we will add a script block to print out each variable. Be sure to make this update in both the Staging and Production deployment stages, and that the Production stage is pointing to the correct variable group. Then push your changes to the repository and review the logs.

The script stage will default to the name CmdLine if you do not give it a specific name. Notice that in the output the variable that we marked as secret did not print out the value? This is how we would want to handle variables that contain sensitive information.

When running the Production step then you will see the same output, except with the values set for the production variable group. The full pipeline with variable groups set in both staging and production can be found here – https://gist.github.com/cashewshideout/fbabcb13f0e395093e8cec812a9b039b.

Templates

This next concept does not require variable groups to be implemented. If you are following along, the examples shown will include the updates from the variable groups section but it is possible to start from the base project of this post.

From my developer experience I have often heard the acronym DRY (Don’t repeat yourself) and been taught about abstracting pieces of code that are replicated. This helps keep things uniform and, similar to using variables, prevents needing to make updates in multiple places. In multi-stage pipelines we have the ability to create templates which allows you to take a section of the pipeline and make it a reusable part. If you have worked with the graphical build or release in Azure DevOps the related feature is Task Groups.

Take a look at the pipeline we are working on now. Notice that the exact same steps are being used in the Staging and Production stages – the only difference between them is the value for WebAppName (Lines 78 – 97 and 114 – 133 https://gist.github.com/cashewshideout/fbabcb13f0e395093e8cec812a9b039b). We would like to make these steps the standard way to deploy this application and don’t want to have them repeated in the pipeline so we will pull out this section into a template.

Let’s create a new folder named templates under deployment to hold our templates and create a file named appdeployment.yml

To start our template, copy the steps from the full pipeline, which will include the download, script, Extract files task, and AzureRmWebAppDeployment task. You will need to start the template with the steps property for context on what is in the template.

In the pipeline.yml file replace those steps in both Staging and Production with a single line referencing the template (line 19 below)

This is the basic setup, but since we copied the Staging steps to the template right now both Staging and Production will end up deploying to the staging web app. To fix this we need to add parameters to the template (it is a similar concept to passing parameters to PowerShell scripts, functions in code, etc). At the top of the template we will define a list of parameters – as I have said throughout these posts, naming is important and the parameter’s name should be short but descriptive. Many times, I end up using something similar to the property that it will be the value for. In this case we will use a parameter name of webappname (keep the casing the same in each reference to the parameter but there is not an overall case requirement).

You can see that we have defaulted the value to an empty string. There are some parameters that could have defaults depending on your company policies but this one doesn’t seem appropriate to have a default. Also notice the syntax for inserting the parameter value in the task: ${{ parameters.webappname }}.

In the pipeline we need to pass a value to this parameter to be used. The gist below shows adding parameters and passing the value to the Staging stage. Remember to also update the Production stage with its web app name.

These updates will not have any effect or changes on the pipeline running in Azure DevOps.

What if, you ask, we want to add a task to Staging but do not want it to run in Production? We can’t update the template because that would affect both stages. Good news, you can add additional steps to the pipeline outside of the template – either before or after.

Here is an example of the full pipeline and template – https://gist.github.com/cashewshideout/a8ce9274fe3daea35566f899c751581e

We only pulled out one value to a parameter but it is possible that you may want additional parameters based on how the template will be used. Creating a few more parameters could make this template a bit more generic and able to share across projects rather than for this specific pipeline.

There are many things you can do to expand on templates that we did not go into here including

  • Storing and referencing templates from a separate repository
  • Using conditions and functions in templates
  • And more

It is a great way to start creating consistency across a single or multiple pipelines.

The End... Or Is It?

In this post you have seen a couple ways to enhance your yaml multi-stage pipelines. Each of these updates are not required for a pipeline to run but depending on the requirements for the pipeline you are building, they can enhance it.

This is also certainly not all that we can add to a yaml based pipeline – there have been a few points that were a little glossed over or that I outright said were out of scope for this series. So, while this official series is over, you may see some additional posts in the future that continue to build on what was started here.

Interested in using Azure DevOps Pipelines for your application?  Let us see how we can help, contact us today!  

Resources

Looking for a faster, cheaper, and easier way to build, manage and maintain a mobile app for your organization? A progressive web app might be the answer.

Traditional native mobile apps like those launched in the Apple or Google Play app stores can be a bit involved to build, manage, and maintain. Depending on the platform (iOS for Apple or Android for everything else), they require deep working knowledge of a variety of highly specialized programming languages—and if you’re planning on releasing the app across different types of devices, you will essentially need to build two separate apps (or figure out a way to translate one to the other by building a cross-platform app, which is almost more labor-intensive). This translates into duplication of development work across code bases and a variety of complications: more cost and more time building, managing, and updating separate apps.

Luckily, there’s a better way.

What is a progressive web app?

To the user, a progressive web app (PWA) looks and behaves just like any other mobile app. It can be accessed by its own icon, and when open it’s virtually indistinguishable from a native app. The difference is that the PWA is running in a browser—which makes it easier, faster, and cheaper to build and support.

Want to see real-life examples?

Starbucks developed this PWA. You can check it out by visiting, app.starbucks.com, on your mobile phone.

We recently worked with the i9 Sports team to redevelop their online presence as a PWA. On your mobile phone visit i9sports.com.

You can also get a preview by viewing a demo of the app.

The “progressive” in progressive web apps comes from the use of service workers, code scripts that can approximate a native app experience by allowing users to access the app even while offline. In the Starbucks PWA, for example, ordering obviously isn’t possible without a connection, but users can still be able to open to the home screen and read the beverage menu—something common in native apps but relatively unheard of in web browsers.

There are a few tradeoffs for choosing a PWA over a native app, but in many instances a PWA is a more effective solution for company and user needs because it is easier to create and maintain. Here are the main benefits.

Progressive web apps save development time

Unlike the different coding languages required for native apps, PWAs are built in JavaScript. JavaScript is a fundamental programming language used by developers worldwide—often the first language they learn after HTML and CSS—and it is used in most web apps across the Internet. Being able to code an app in JavaScript rather than Objective-C, Swift, Java, or Kotlin (or a combination of those and others) saves a substantial amount of developer time.

Another benefit is that in a progressive web app, there is one code base for the app, rather than separate code bases for iOS and Android versions. That means one set of updates rather than multiple and less troubleshooting across platforms—and as PWAs are web based, updates to the app happen automatically and don’t require users to download them to access the latest security features or improvements.

Progressive web apps are more accessible and easier to launch

It’s not easy to launch an app in the Apple App Store. There is a tedious approval process with high (and, some would argue, arbitrary) standards for acceptance, along with high competition with other apps. In addition, having to visit an app store (iOS or Android) to download an app adds another required step in the buyer journey. Requiring potential users to visit an app store and follow through on their intent to download the app creates friction and may lose potential users in the process. By contrast, a progressive web app can be downloaded directly from anywhere—a company website, an email attachment, or anywhere in the cloud.

To be fair, there are a few tradeoffs that make native apps a better choice for app store launches. If your app depends on integration with other native mobile apps (such as calendars or social media apps like Facebook) or in-app purchases that would normally be handled through an app store, a native app might be a better solution. And if your heart is set on launching in an app store, a progressive web app adds some complexity to that process as it requires extra steps for approval.

But for internal apps and a large array of customer-facing ones, the benefits of an app store launch don’t outweigh the drawbacks of an easier development process and a simpler launch process. In these cases, a progressive web app is a practical, lightweight solution to your needs.

Progressive web apps save your company money

Both above advantages are benefits in themselves, but progressive web apps also translate into lower costs for development and maintenance. Time is money in many aspects of business, and software development is no exception. Why build a native app in multiple codebases, resulting in up to twice the development costs, for no real advantage? Why make your app more difficult and expensive to update and maintain, requiring even more developer or DevOps professional time, when you could invest in a process that’s easier to support internally?

The truth is that if a more lightweight progressive web app can accomplish what you and your users need, it is a far wiser, better use of company resources. By saving developer time and simplifying the process, your company could benefit significantly in terms of cost savings and ROI. And because PWAs are easier to maintain and extend, that time and money saved extends into the future with quicker, less complicated updates.

Interested in determining if a progressive web app or a native app is the right fit for your organization? At MercuryWorks, we can guide you through each step of the process and advise you on what type of development solution will work best for you. Reach out to us today.

Progressive Web Apps are just websites that took all the right vitamins.

What is a Progressive Web Application?

Progressive Web Apps (PWA) are websites that use modern web technologies to deliver experiences closer to those delivered by native iOS and Android apps.  They combine the last decade’s browser and web construction advances (responsive web designJavaScript enginesservice workers and web storage) to create a whole greater than the sum of their parts – a capable substitute to native applications.

At its core a PWA is still a website but can be installed on your device (phone, tablet, PC or Mac), app icon/shortcut and all.  A PWA will pull updates in the background each time a user runs it.  As you update your web application, not only will visitors to the app site gain access to new features, PWA users will also get the updates – all without explicitly performing any kind of update.

When a user interacts with the PWA, the PWA can check the capabilities of their device and load more data in the background.  This opens up scenarios of preparation for coming interactions, resulting in the appearance of a more capable application. PWAs can also offer app-like increased functionality traditionally unavailable to websites – offline usage, notifications when new data is available and various other conveniences.  This progressive unfurling of functionality based on the user’s device works on the same code base and in parallel with users of your website in a browser.

Why a Progressive Web Application?

User experience research shows that while users turn to native apps because they provide a cleaner and smoother experience (as compared to websites), mobile websites are their first port of call for exploring a topic and for seeking specific information when on the go.  While certain modern app features like ability to work offline and subscribe to notifications have not traditionally been present for websites, their findability via search engines contributes to their preferences for information retrieval.

One factor driving the affinity for native apps by mobile users is that native apps are de facto created mobile-first from the start.  While “mobile-first” has been a design mentality for websites in many circles for a few years, it is not universally practiced.  Thus we arrived at the “there’s an app for that” reality: native apps delivered what mobile websites couldn’t – the ability to work offline and an experience optimized to the mobile form factor.

On the pro-web end of things, we encounter the inherent friction of native app distribution to end users.  While distributing software to end users via a centralized app store is convenient for developers, it imposes work on users.  Store providers impose a review period on new releases, users can be running any of your past versions (tech support goes way up there) and in the worst case the app store may even reject an app entirely.  Finally we encounter friction on the part of creators for both initial development and updates for several versions of native apps for different operating systems and form factors.

From the dawn of the web, this process has been greatly simplified: instead of downloading and installing, you typed in a URL or clicked on a link.  When you had a new version of your app, new content or altered layout you published your website and your users got it right away (and only one version ran at a time!).  This was the vision of proper software-on-demand that in some ways the advent of native apps regressed on.

Thus the industry came together (Apple begrudgingly so) to find a way to marry the immediacy and low entry barrier of the web with the form-factor optimization and functionality of native apps.  The “why” of PWAs became:

  • To provide a user experience on par with native mobile applications

  • To increase speed and cost of development as compared to by-device native development

  • Provide connectivity flexibility: work offline or on low-quality networks

  • Eliminate app installation friction, avoid app stores’ “walled garden”

  • Eliminate version release and support headaches

  • Create re-use across desktop, phone, tablet and other form factors

  • Improve upon typical website re-engagement

  • Provide more timely user information with service workers and new web APIs like the Web Push and Notifications APIs

What makes for a good PWA?

In my experience, native apps have less features than the same company’s website/web app but carries them out extraordinarily effectively: they’re very streamlined and tailored towards the user’s “job to be done“.  Also, it’s there on prime real estate: the user’s phone screen.  To be competitive to this experience and meet user’s conditioned expectations, a PWA needs to be:

  • Discoverable – a PWA is available on the web as well as in app stores
  • Installable – the PWA displays alongside all the other apps (start menu, icon on homescreen)
  • Responsive – an interface catered to the environment, offering smooth and immediate interaction
  • Network independent – functions both offline, online and on flaky connections
  • Fresh – updates happen automatically with no concerted installation process
  • Safe – data delivered over secure channels
  • Linkable – accessed via a simple delivery mechanism, such as a link (including deep links into specific sections of the app)

When it comes to PWA construction, “FIRE” is a convenient acronym to keep in mind:

  • Fast
    The first contact with a PWA interface is an immediate, responsive and enjoyable experience
  • Installable
    The PWA installs like any other native app, showing on your desktop or homescreen rather than requiring a browser
  • Reliable
    The PWA is available and responsive every time the user opens it, on par with any other piece of installed software
  • Engaging
    The PWA follows the best practices for high-quality UX and works irrespective of network state or device capability – there is never an empty screen telling the user to go online or upgrade

Are PWAs supported?

Many of the piece-parts that make PWAs possible have been in existence and enjoyed some amount of browser support for years, especially from Google Chrome.  Certain key browser supports to make them a viable wide-band solution to rival native apps.  Apple’s Safari browser’s notable lack of support for Service Workers and Web App Manifests effectively put the iPhone and iPad ecosystem out of reach for a PWA.  Thus for years prior to 2018 the PWA was nullified as a viable alternative to developing platform-targeted iOS and Android apps.

Thankfully, that is behind us and now, for the vast majority of users, PWAs are a highly viable and supported option: all told, as of August 2020 PWAs are supported for about 95% of worldwide users.  The following details break down the browsers and OS versions that, as of August 2020, fully support PWAs:

  • Windows
    • Edge 17 (legacy Edge)
    • Edge 83+ (Chromium Edge)
  • MacOS
    • Safari 11.1+
  • Both Windows & MacOS
    • Chrome 45+
    • Firefox 44+
  • iOS
    • Safari 11.3+
  • Android 5+ (Lollipop)
    • Android Chrome
    • Samsung Browser

For IE11 and other ancient browsers (even Microsoft is abandoning IE11) the PWA will work in a typical website fashion - only PWA features of offline support and web notifications will not work. That is a failure mode I can live with.

What makes for a solid PWA strategy?

First the good news: the first step in any PWA strategy is to build a modern website that follows a host of best practices:

  • Responsive web design principles (ensure high fidelity layout for mobile, tablet and desktop)
  • Highly performant (you want the first experience to be fast and smooth)
  • Be as small as possible (eventually this will be installed on a user’s device)
  • A high level of accessibility (Google Lighthouse is a good tool to gauge 508 compliance and overall performance)

A key mental framing to keep in mind is that for likely the first time in your team’s careers, their users will be using installed software outside the confines of a browser.  The moment you remove a browser’s UI (and attendant UX features), users will assume a superior level of functionality so be ready to not only deliver on that expectation but also avoid the crutches a browser’s chrome provides for you.  Said another way, there are no back buttons, refresh button and other niceties that browsers provide.

Here are few fundamental considerations for a PWA that diverge a bit from a typical “best practices” web application:

  • Think About UI Two Ways
    Mock screens with and without the browser UI to help uncover any potential layout issues.  And remember that some users will be using your PWA as a web app in a browser (not all users will be using it in installed mode).
  • Plan for Different Connectivity States
    Remember to design for different connectivity states: online, offline, flaky network, content loading and content failing to load.  One of the more complex design patterns is sync’ing state and content after loss of network connectivity.
  • App Icon and Start Screens
    Not only should the app icon and starting state screen be attractive and on-brand but also consider each different OS and provide an icon for each (nope, they aren’t all the same size and shape).
  • Asking Installation Permission
    Think about when you will want your app to prompt the user to add the PWA to their home screen/desktop.  Asking too soon is definitely pushy and bound to failure but asking too late and they might not bother.  My general advice is to wait until the user has done enough in the app to be bought in but not so long that they’ve just bookmarked it.
  • Asking for location data
    Don’t just ask for permission to access the user’s location as soon as they open the app – wait to ask for permission until their is a purpose-contextual reason to ask for this level of information and provide value for doing so.

Finally, before going overboard with every feature now possible with a PWA, decide what “jobs to be done” a user is looking to accomplish.  A well-designed PWA (and any digital product) should make it easier for users to complete their goals.  Some framing considerations:

  • Have a clear and memorable value proposition in a prominent position – make sure the user knows what they can do
  • Make procedures as smooth as possible; users tend to have little patience completing forms and finalizing the checkout process, particularly on phones and tablets
  • Be careful with push notifications – they can increase engagement but carry more impact if delivered at the right time (in other words, not as soon as they hit your home page)

Some fine points

There are a few fine points mentioning before I show you some highly successful PWAs and provide you resources to get going with your first PWA.  The following are tactical points to be aware of as you start your PWA journey:

PWA Branding and Device Icon
  • Add To Home Screen
    • Android Chrome will prompt the user to add the web app to their home screen
    • iOS Safari doesn’t prompt, but we can add our own prompt
  • Branding
    • We add client-branded images for app icons and the loading/splash screen to give the app a native feel (Android has more options, iOS is more limited)
PWA Notifications
  • Web notifications are only operable while the user is online
  • We can send web notifications to users on supported desktop browsers
  • PWAs can send native push notifications to Android devices with the PWA installed as an app
  • PWAs cannot send native notifications to iOS devices or while the app is not being used
    • To overcome the iOS limitation, we can send our own designed notifications while they are using the app
    • Finally, as a full solution option, we can create our own notification service using Web Sockets or a service like Google Firebase
PWA Offline Use Considerations

Questions that need to be answered answered before implementing a PWA:

  • Do we need to store any data offline?
    • If users are primarily searching, we may not want to store data offline
    • Data looks to change often, if not daily, so keeping data current is a concern
  • Do users have good WiFi or LTE connections?
  • How will the app work with authentication and security/logging in?

Resources around offline support (provided this is a key feature for your app):

    • Navigator.oneLine API
      • Built-in browser support for detecting of a user is online or offline
    • Local Forage
      • Helper library for browser compatibility for storing data offline in browsers best supported database
    • Workbox
      • Google-backed JavaScript libraries for adding offline support to web apps
PWAs aren’t just for smartphones

The vast majority of literature and general coverage about PWAs concerns their use on smartphones.  A key point to keep in mind is that PWAs are also extremely powerful on the desktop as installable software for your users.  While having an app icon on your users’ smartphone is extremely powerful, so is having a shortcut in their Mac or Windows taskbar!

Examples

While the possibilities of PWAs are still new, several leading organizations have begun to innovate and notch some highly successful outcomes.  The following are three major PWA accomplishments that have hit my radar:

  • 3-4x faster load times
  • 50% increase in mobile users
  • 4x increase in conversion rate to sales site
  • 84% decrease in load time
  • 17% increase in conversions
  • PWA mobile traffic now greater than desktop
  • 50% increase in mobile sessions
  • 150% greater engagement from PWA installers
  • Covers 55 countries

Some PWA resources

If you’re compelled with what you’ve read so far, you’ll want to dig into some tools and other resources to learn more of a technical nature and potentially dig into your first PWA.  Dig in:

Tools
  • Lighthouse: open-source automated tool from Google for improving the website quality.  Can be run against any web page (public or private).  Provides audit scores for performance, accessibility, PWA compliance, and SEO.
  • PWA Builder: open source project from Microsoft to pre-seed a PWA manifest from an existing URL and create a ServiceWorker for you.  Includes fallbacks for instances where PWA features are not supported.
  • PWA Asset Generator: a developer-friendly CLI tool to help create PWA icons and launch images
Websites and blog posts
 Videos