• Enhancements to general multi-threading and exception handling to provide better stability and performance of the build system
• Changes for improved .NET stability, including better support for referencing C# projects without project GUIDs, and moving projects across solutions
• Improved Telemetry, designed to proactively catch unexpected exceptions and make it easier for us to diagnose issues and thus provide more effective support. When you contact our Developer Support team, provide your Telemetry IDs by navigating to Tools > Options > Synergy DBL > Advanced (see image below). These randomly generated IDs will enable us to look up the diagnostics for your specific session to troubleshoot the issue. (The telemetry data is anonymized.)

• Ability to load Common Properties and Environment Variables set from a selected project in the Solution Explorer into the Synergy VS Command Prompt (see image below)

Note that Winforms and WPF projects are now deprecated (see image below). We’re no longer maintaining these project types with Microsoft updates. Winforms and WPF templates will be removed in a future SDI release, but the projects will continue to load until a Microsoft update fully breaks their support. We encourage you to turn on Telemetry to help us gain insights into whether these projects are still being actively used.

The improvements in this new release will provide you with a more seamless development experience. We encourage all users to upgrade.

Your feedback is crucial to us. Please share your thoughts on this release or anything SDI-related by emailing software development manager Tim Bauguess at tim.bauguess@synergex.com. We would love to hear from you.

The post Announcing SDI 2023.04.1150 appeared first on Synergex.

Constellation Software, Inc is a publicly traded Canadian company. Just as Synergex is committed to helping you preserve your legacy software, FOG/Constellation is committed to helping companies “preserve the legacy of their businesses”. They have an “Own For Life—Never Sell” philosophy. Their focus is on strengthening and growing businesses over the long-term. I’m confident that their expertise and resources will help ensure that Synergex will continue to thrive far into the future. Plus, we’ll have the opportunity to do bigger and more exciting things. 

Although we’re now owned by FOG, we’re still Synergex. As a customer, you probably won’t notice many changes. FOG has a decentralized business model—their acquired companies typically run autonomously. I envision that Synergex will continue business as usual. The existing Synergex leadership team (including myself) will continue to operate and grow the company.  Taking this leap and joining forces with FOG/Constellation has reinforced my vision that your Synergy applications will still be running—and thriving—in the year 2100 and beyond. If you have any questions about this change, please reach out to me, other leadership team members, or your account manager.

The post Synergex Acquired by FOG Software Group appeared first on Synergex.

• Additional multithreading stability improvements to reduce intermittent build, debug, and project load issues
• Improved editor performance
• IntelliSense and Build process enhancements, including support for spaces in names for folders with source files for Multiple Mainline projects

View the release notes for detailed information about the changes.

The post Announcing SDI 2023.03.1080 appeared first on Synergex.

• When the product needs to be part of a larger bundled set of products
• When additional operations (secure signing, testing, staging, etc.) take place after the product is built
• When the product is developed in a continuous integration/delivery (CI/CD) system, or when there is a goal to move to a CI/CD system

In this article, we will focus on the last of these, building in a CI/CD system, which in simplified form looks something like this:

Consider the Build step in this scenario. Building from the IDE is not ideal for CI/CD because in a fully automated environment, the result of a build should be immediately consumed by a testing infrastructure. Additionally, with CI/CD it is generally best for builds to occur in a separate environment—i.e., in a dedicated build environment. However, a build environment may not be able to launch the IDE due to machine settings, resources, licensing, or other environmental limitations. But if IDE builds are not ideal for CI/CD, how should the product be built in this situation?

Enter the MSBuild command-line interface (CLI).

MSBuild is Microsoft’s engine for building applications and libraries from Visual Studio projects and solutions. It is included in Visual Studio, and it runs behind the scenes when a build is invoked from Visual Studio. However, MSBuild is also available as a stand-alone product with a CLI that provides command-line equivalents for Visual Studio’s build functionality. Given a Visual Studio project or solution file, MSBuild can build the product from a single command. This solves several CI/CD-related issues:

• Because the stand-alone version of MSBuild is portable and can operate without Visual Studio, it simplifies requirements for machines dedicated to product builds. There is no need for Visual Studio licenses or extensive resources. This is especially useful for automatically provisioned machines in the cloud or in a pipeline environment.
• There is a stand-alone 64-bit version of MSBuild, but only the 32-bit version is available from within Visual Studio. So if you need the 64-bit version for a build, the CLI is your only option.
• The MSBuild CLI is automation friendly. It is much easier to automate a command-line build than to configure automation for a Visual Studio build.
• A command-line build can easily be run as part of a larger automation process that builds multiple products and bundles them or prepares them to be consumed in subsequent automation processes (signing, validation, testing, etc.). The same applies when a product has other requirements, such as environment setup or dependencies that require other components to be built prior to the product build. A command-line build is also ideal if a product has pre- or post-build processing requirements.

To illustrate how MSBuild CLI can be part of a larger CI/CD cycle, the following shows an expanded view of the Build step in the previous diagram:

As you can see from this diagram, an IDE build would not fit well as a replacement for “MSBuild CLI build,” particularly in a fully automated environment.

We have been discussing why the MSBuild CLI should be used for product builds in a CI/CD environment. Now let’s examine how it is used. The stand-alone version of MSBuild comes as an executable that supports many build options through command-line switches. We will cover some of the most commonly used options below, but first let’s discuss how to install MSBuild as a stand-alone product. The stand-alone installer is available on the main download page for Visual Studio:

https://visualstudio.microsoft.com/downloads/

Look for “Build Tools for Visual Studio 2022” on the page, or use the following direct link (subject to change):

https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=BuildTools&rel=17

Once you have installed MSBuild, there will be an msbuild.exe file at the location you chose for the installation. This file is the MSBuild executable, which you’ll specify at the command line. In the most basic case, you can specify just “msbuild” followed by the project or solution file you want to build. For example:

msbuild MySolution.sln

This command will build a solution named MySolution using the default configuration and platform for that solution. (The default is usually the “Debug” configuration and the “Any CPU” platform.) Again, this is the simplest form of MSBuild command. To exercise more control over a build, you can use command-line switches to specify build behaviors. The following are some commonly used switches. MSBuild supports ‘/’ or ‘-’ for switches, but for simplicity’s sake, we’ll use the ‘-’ form in the following.

• -property:name=value or -p:name=value

The -property (or -p) switch enables you to provide or override an MSBuild property. The most common use for this is to select a specific configuration or platform. For example, the following will produce a build based on the Release configuration and the x64 platform:

msbuild.exe -p:Configuration=Release -p:Platform=x64 MySolution.sln

Of course, this switch is not limited to just configuration and platform. It can be used to specify any MSBuild property. A common use for -p when building Synergy projects is to specify the solution directory for the project, since some of the SDI tooling relies on the SolutionDir property setting when building a particular project instead of the solution. For example:

msbuild.exe ‑p:Configuration=Release ‑p:Platform=x64
‑p:SolutionDir=”C:\dev” MySynProj.synproj

• -maxCpuCount[:number] or -m[:number]

The -maxCpuCount (or -m) switch specifies the maximum number of concurrent MSBuild processes allowed when building. If no number is specified, MSBuild uses the number of processors in the computer as this maximum. This feature allows for parallel builds of the projects in a solution, which results in faster overall build times.

• -target:targets or -t:targets

The -target (or -t) switch specifies MSBuild targets to run. If this switch is not specified, MSBuild will run the “Build” target for the specified project/solution. For example, if you want to rebuild the solution instead, use this switch to specify Rebuild. For example:

msbuild.exe -t:Rebuild MySolution.sln

This switch also supports multiple targets in a semicolon delimited list. So, if you want to be explicit and specify a clean and then build (instead of specifying a rebuild), you would use a command like the following:

msbuild.exe -t:Clean;Rebuild MySolution.sln

• -verbosity:level or -v:level

The -verbosity (or -v) switch enables you to control the level of build information to be displayed or logged. Options are q[uiet], m[inimal], n[ormal] (which is the default), d[etailed], and diag[nostic]

• -graphBuild[:True or False] or -graph[:True or False]

When set to True, the -graphBuild (or -graph) switch causes MSBuild to construct a graph identifying project dependencies. MSBuild will also attempt to build project references prior to building the projects that reference them. (This is supported only for MSBuild 16 and higher.)

The full list of switches, along with more complete switch documentation is available at https://docs.microsoft.com/en-us/visualstudio/msbuild/msbuild-command-line-reference?view=vs-2022.

To recap, the MSBuild CLI is ideal for building products in a CI/CD system or in any fully automated system. The MSBuild CLI provides benefits unavailable with a Visual Studio build, including a more automation-friendly interface, fewer machine resource/licensing requirements (because it is a stand-alone tool), and easier set up for pre- and post-build requirements.

We hope this article encourages you to use the MSBuild CLI in your build environment and to adopt automation or CI/CD in your product development process. As always, we are excited to hear from you. If you have any questions or comments, contact Synergy/DE Developer Support or post in the Community area of the Synergex Resource Center. We will be happy to answer your queries!

The post MSBuild CLI: A Gateway to Continuous Integration/Delivery appeared first on Synergex.

Hire Right and Welcome Them

Much of the success in hiring depends on finding the right candidate. To increase your odds of success, give developer candidates a proficiency test to ensure a level of technical proficiency. During the interview, try to gauge the person’s interest and enthusiasm for the subject matter. Learn about their personality and decision-making processes. Will they fit with the culture of the company? Will they get along with you? Is there anything unusual that stood out about the candidate that you liked?

First impressions count! Before your new hire arrives, devise a welcome package for them. If you don’t have one, contact your Synergex account manager for examples. Most importantly, before they arrive, create a specific 90-day plan that details all the training, tasks, and goals for each 30-day interval. For each item, list the goal and the one or more tasks associated with that goal. In addition, include tasks they can do when there’s downtime, such as learning about the company itself. Training on languages, tools, and processes are all important. Consider also what training and classes they’ll need during each time interval, so that by the end of the 90 days they have a good base of understanding on which they can build additional knowledge. Developing a 90-day plan tells your new hire that you have thought about them, that they are important to you, and that you want them to succeed. After each 30-day interval, do a performance review to go over the expected completion of tasks. If you’re looking for a sample 90-day plan, contact your Synergex account manager and we’ll be happy to forward one of ours.

Document the Unknown

The base design of the current Synergy compilers was developed over the course of 15 years by a large group of developers. At Synergex, we typically document software with extensive comments. But throwing someone into a code base, even with extensive comments, is not fair. You may have old requirements documents and thousands of pages of old design documents with pretty diagrams, but, if they’re not already out of date, would you really want to sit down read that volume of documents? Please don’t bore your understudy.

You understand years of decisions and designs about that software. You know its strengths and weaknesses—so share these in a concise, readable form by developing a “My Software 101” document that gives bite-size descriptions of each important individual design concept in your software. Describe these in common, easy-to-understand terms so that a person without your level of experience can grasp them. For each design concept, ask yourself: What is its purpose? Why is it important? How does it fit in with the whole design? Add detail as necessary, but don’t get into the weeds—stay focused on the higher-level concepts. For example, in our 101 document on the Synergy compilers, we broke the design down by stages of compilation first, and then went into more detail on the main concepts in each stage.

Code Reviews with a Purpose

Let the new hire read the 101 document and ask questions. These questions will guide you to improve the document, so take the opportunity to do so.

Using the 101 document you developed, on a frequent basis (we do daily), review the code associated with a single bite-sized design. Before COVID-19, we sat side-by-side to review code, but we’ve found that remote presentation over Microsoft Teams also works well. Start with the most basic building blocks of the design concepts and work your way into those that are more complex and that build upon others. Explain how the code implements the design and what is important to understand about the code for a particular design.

Small Challenges

At some point, you’ll want to give your new hire some small challenges, likely during the last interval of the 90-day period. It’s best if you can find a bug related to some section of code you’ve already reviewed with them. Find something you think they should be able to solve on their own given their training up to that point, and which isn’t time critical.

Once you assign the challenge…step away! Let them try to solve it on their own and have them contact you when it’s done. It’s likely their solution may not be exactly what you would have done. That’s OK. Review the code to see whether it meets your coding standards, is maintainable, and solves the problem. If there are improvements that could be made or a different approach that could be taken, ask questions about their solution to guide them toward these changes. The important part isn’t to tell them how you would solve it or exactly what they need to do, but to guide them in how to think about the problem, so they can come up with the solution on their own.

As your new hire completes these small challenges, you may find additional design concepts that you hadn’t thought of initially, or things that you thought were too detailed, when creating the 101 document. As you find these, add them to the 101 document and do a review of them along with the associated code.

The Deep End

After your new hire completes the 101 discussions and reviews, along with a number of smaller challenges, you’ll want to give them something much more challenging. Maybe it’s a mini-project they can do on their own or a project where they’ll need to work together with other developers. Whatever it is, make sure the tasks and expectations are well-defined and understood. Set a schedule for the completion of tasks and let them know you are available to answer questions. Again, as before, use leading questions to develop their ability to solve problems on their own.

Throughout the project, ensure you touch base with them at least once a day to find out what they worked on yesterday and what they plan on doing today. Not only will you get a sense of how the larger project is going, but it’s a good time for them to ask any additional questions of you and promote the team dynamic.

The Result

The goal at the end of it all is to have an efficient, knowledgeable developer who understands the important design concepts used in your software and can think on their own of solutions to problems they encounter. Much of the success of it does depend on the talent and ability of the new hire, but you also play an important role in that person’s success by mentoring, guiding, and encouraging them along the way. By planning from the start, you can make this a reality for your team and your company.

The post Training Others to Continue the Job appeared first on Synergex.

Given enough time and enough money, it may be possible to capture the decades of nuanced business knowledge, best practices, and mission critical logic that have been crafted into your legacy (aka proven) enterprise code. However, over many years of working with enterprise IT organizations and their CxOs, I’ve seen very few of these projects completed, and I’m not sure if any of those would really be considered “successful.”

In the best-case scenario, the organization recognizes that they’re going down the wrong path early in the effort (often when new CxO #2 enters the picture), and they return to their trusted legacy code. They discover the logic of moving to a modern solution with the code that has kept their business running for decades. Add a modern UI/UX? Yes! Integrate with other systems inside or outside your business? Sure! Go to the cloud? Not a problem! The trick is to add whatever front-end or new functionality you want—while keeping the proven legacy code that runs your operations.

In the worst-case scenario, they continue to run away from their legacy stability as they pour money and time—many years’ worth—into attempting to recreate something that was never broken. The legacy code that keeps businesses thriving has been crafted over the years to the point where just understanding all that it does—and why—would be an extremely challenging, multi-year project in itself. And heaven help a development team that sets out to replace a legacy system without knowing everything that it does. I believe this is the primary reason these projects fail: they require much more time and money to replace functions the project team never fully understood. (For more on this, see my blog post titled “When It Comes to Legacy Systems, It’s Hip to Be Square.”)

$500 Million Trainwrecks

I’m not the only one who has seen the trainwrecks that can happen when perfectly good legacy code is tossed out simply because it’s perceived as being old. (Age discrimination for applications?) Henrico Dolfing, who spoke at our 2022 DevPartner Conference, maintains an entire blog on this topic. Or you can find your own: Google “SAP disasters” or “ERP project fails” or whatever similar phrase comes to mind.

A few years ago, The Register published an article headlined “ERP Disaster Zone: The Most Costly Failures of the Last Decade.” Just one example from their collection: A German supermarket chain spent 500 million euros—about half a billion dollars—to replace its in-house legacy merchandise management system . . . before terminating the project. To the company’s credit, they finally pulled the plug at the half-billion-dollar mark, while others remain trapped by an “in for a dime, in for a dollar” mentality of hoping that success will be somewhere down the road, without realizing just how long and expensive such a road can be. But wow! Half a billion? That’s a lot of money down the drain. And, had the plug not been pulled, it could have been even worse.

The Costs Keep Coming

In my many years of experience, I’ve seen that organizations attempting to replicate the functionality of a legacy application in a new application not only spend significantly more years and many more millions of dollars than initially planned, but they also end up with an application that differs greatly from the plan. As reality beats away at their original aspirations, the glowing vision slips into survival mode: What can we do to keep things at least business functional?

Even if and when the development efforts are declared complete, the costs keep coming.

A legacy system generally has a low base for operational costs. You are dealing with a known quantity. Even a system’s shortcomings are known and can be addressed. But with a new application, you are continually dealing with unknowns, as shortcomings arise because (despite all the time and money invested) the new application doesn’t capture everything that was crafted into the original system.

This means operational costs are higher, and the work-around development costs never really end.

A key question for these “finished” projects is, considering all the costs during the project and the additional costs now required to develop/maintain/support the new system, does the ROI really pan out? Will the new system have enough benefits to warrant all that extra time and money? Especially when you could have moved forward and met your modernization needs at a much lower cost and much less turmoil with your proven legacy code?

One former customer offered a candid reflection on the question of ROI based on his company’s experience. He noted, “Moving to the new ERP system took seven years instead of the planned three; the budget increased by 500%; and we still don’t have the stability or speed of our legacy system.” (For more details about lessons he learned on that project and others, I suggest you read the case study.)

Invest in (and Move Forward with) What You Have

From my experience, here’s how the typical scenario plays out: The legacy (again, the proven) enterprise application that was developed and refined over the years has been fine-tuned and tested and is a rock-solid product servicing the needs of the company. Any issues are known. These may include a desire to update to new hardware or operating environments, move to the cloud, enhance the user interface/user experience, improve data access, provide web access, find skilled resources to continue development and maintenance, and so on.

Yet most companies do nothing to address these known issues—essentially kicking the can down the road. The reasons vary: inertia, thinking that nothing can be done to improve the situation, no one championing a way forward, an “out of sight, out of mind, if it ain’t broke, don’t fix it” mindset, no budget allocation for enhancements, lack of domain knowledge because the original creators and developers have moved on, technology influencers looking for the next shiny toy to add to their skill list or thinking they can save the company money by reducing license and other fees with “their” solution, and many more.

Then one day, the company decides changes are needed. A big budget is allocated, a vision of a perfect new system to replace the old one is presented, project schedules are established, and the project moves forward. Interestingly, these projects used to be projected for two years, and it was fairly easy for companies to pull the plug when they went awry. These days, initial projections for such projects are typically three to four years, though in reality they take seven to ten. With that larger investment, companies have a harder time pulling out when they realize they’re on the wrong course. They feel there’s no turning back, so they throw even more time and money at the project and continue down an unfortunate path.

A new CxO might go into this type of project thinking this predicament could never happen to their company, believing they are doing it differently, but the same outcome occurs time after time. The success rate of such replacement projects is consistently very low.

My advice: Rather than spending a massive amount of time and money creating something new, simply enable your existing legacy system to meet your needs—whether it’s adding a new UI, moving to cloud-based infrastructure, generating user-friendly analytics, or whatever other advancement you’d like. You can be a hero. It will look great on your annual review: Saved the company $500 million and 10 years of misery.

Three Questions to Ask

In an ideal world, the following questions would be thoroughly researched and objectively answered before anyone considers heading down a replacement path:

1. What are the real problems we are trying to solve?
2. Can we solve these problems using our current system?
3. What are the actual costs of alternative choices (money, time, stability, loss of functionality, distractions, the duplicated effort required to continue maintaining an existing system while developing the new one, etc.)?

With almost any legacy system, vendors are available with tools and resources to help achieve the desired results at a fraction of the cost and with a high degree of success. Add a new UI/UX, put your application in the cloud, give data access to executives, train the next generation of developers, have managed services to do the work for you—companies (including many Synergex customers) are doing these things and much more with their legacy systems.

Wouldn’t it be a relief not to have to replicate decades of business intelligence? Knowing that you have it in place and can focus your efforts on extending that critical logic instead of replacing it, you’ll have the best opportunity for a successful modernization project, one that requires a lot less money and provides a superior outcome. Synergex’s investment in our tools and consulting services has a proven track record of ensuring success. In our 47 years, we have never wavered from our primary focus of empowering you to leverage your initial investment in perpetuity. Or as I like to say, your legacy-based applications will still be standing—and thriving—in the year 2100 and beyond.

The post Why Legacy Systems Remind Me of That Song “I’m Still Standing” appeared first on Synergex.

This release includes several key enhancements and fixes to improve your developer experience, including

• Multithreading updates, which improve the stability of the product, reducing intermittent build, debug, and project load issues that some users have experienced, particularly around .NET projects.
• Fixes for unexpected build task failures, including DBLT and DBPT; these improve error list window reporting to make it easier for you to quickly identify and fix these issues.
• General IntelliSense improvements to provide you with more accurate and useful suggestions as you code.
• A variety of compiler updates to support new .NET language features and improve compiler quality.

More multithreading enhancements are planned for upcoming releases. We believe our focus in this area will make a significant difference to your developer experience, providing you with a smoother, more efficient process. View the release notes for detailed information about the changes in this release.

The post Announcing SDI 2023.3.1019 appeared first on Synergex.

New downloads for the REV11 Licensing Upgrade Package and Local Synergy/DE Product Documentation are also available with this release. View the release notes at the bottom of the Downloads page for detailed information about the changes.

The post Announcing Synergy/DE 12.1.1.3287 appeared first on Synergex.

When Azure pipelines were first introduced, developers created and configured these pipelines exclusively via GUI screens that presented CI/CD operations as GUI options. Here’s an example of a GUI screen that enables developers to choose operations for an Azure pipeline:

This approach can work for basic tasks, and developers can still use these GUI screens to configure Azure pipelines. But these GUI selections are not always interpreted correctly when pipelines are generated, so resulting pipelines do not always work as expected. Fortunately, as mentioned above, you can also set up Azure pipelines using YAML, which is a data serialization language (like JSON). Depending on whom you ask, YAML stands for either “Yet Another Markup Language” (its original meaning) or “YAML Ain’t Markup Language”. In any case, for some time now Azure DevOps has enabled developers to define pipelines by coding them in YAML, and Microsoft now recommends this method. This not only eliminates issues resulting from imperfect pipeline generation from GUI screen selections (one less variable when a pipeline fails), but it gives developers more control over pipeline operations and flow.

In this article, we’ll briefly look at the basic steps for setting up a simple YAML pipeline that builds, tests, and deploys. But first, let’s take a closer look at CI/CD. In simple terms, continuous integration (CI) is the automation of builds and testing every time a developer commits a change to version control. And continuous delivery or deployment (CD) is the automated deployment of built software to a test or production environment. CI/CD processes can be complicated, and that’s where YAML comes in. YAML enables you to code complex operations (e.g., multiple CI/CD workflows) in a single file, so that they all operate as one contiguous workflow, one pipeline. For example, once a build is completed and tested, a multi-stage YAML pipeline could then deploy the build to different environments for further testing before finally deploying to production. The figure below shows Azure DevOps reporting on the build and deployment stages of a simple pipeline we have for an internal service.

Before you start…

There are some prerequisites for creating Azure pipelines. You must have access to Azure DevOps with permissions to create pipelines, and you must have a deployment environment (e.g., a web service) for your pipeline to deploy builds to. Additionally, the source code for the software you are developing must be stored in a version control system. In the example pipeline we’ll look at in this article, we’ll select a Git repository in Azure, but you can use some other version control system such as Bitbucket or Subversion.

Creating the base YAML for a pipeline

Now let’s see how YAML is used by creating a simple YAML pipeline. We’ll start by instructing DevOps to set up a basic YAML file that will be the starting point for our pipeline and will be added as a .yml file to the repository we’ll select in this process.

After signing into Azure DevOps, we’ll select Pipelines in the navigation area, select the Pipelines option under that, and then click the “New pipeline” button in the upper-right of the Pipelines area.

Next, we’ll specify where the source code is coming from. For this example, we’ll use the “Azure Repos Git” option (the first option shown in the following screen capture), and then we’ll select the repository that the pipeline will be generated for. (Azure DevOps prompts you for this once you’ve selected the version control source.)

We’ll then select the type of project we want to use the pipeline for. The example pipeline we’re setting up here will deploy to an Azure web service, so we’ll select ASP.NET:

At this point, DevOps creates a basic YAML file from the choices we’ve made, and it displays that file in Azure DevOps:

Understanding YAML basics

Now we finally see what a YAML file looks like. We can add more code to this file to define pipeline operations, but first let’s break this starter code down a bit and see what it does by looking at some of the keywords it uses. For more information on YAML, including YAML keywords, and for information on Azure pipelines in general, see Microsoft’s documentation on Azure Pipelines.

trigger:
The trigger keyword at the beginning of the file causes the pipeline to run when a specified event takes place. In this case, the pipeline will run whenever an update is pushed to the specified repository branch, which is “main” in the code shown above.

pool:
The pool option specifies the agent that will be used to run pipeline operations. For the base pipeline we just set up, it defines the specs for the virtual machine image, ‘windows – latest’, that will be used to build the project. As of this writing, this is a Windows Server 2022 image with Visual Studio 2022. Other options include Ubuntu and MacOS virtual machines. 

variables:
The variables keyword enables you to set system variables or define your own variables at the beginning of a YAML pipeline file. These variables are treated as environment variables when the pipeline is triggered. Most variables are defined using the format solution: ‘**/*.sln’. And once they are defined, you can generally use the following format to reference them elsewhere in the pipeline: $(solution).

stage:
Stages, which are defined by the stage keyword, delineate the larger divisions in a pipeline (and come after the stages keyword). For example, the following code is for a build stage. It is important to place tasks for a stage under the job section for that stage (e.g., job: Build) to instruct Azure DevOps that they belong only to that stage.

Likewise, we can add a deployment stage:

task:
Tasks define what the pipeline should do. The steps keyword must precede the task keyword, or you’ll get an invalid YAML structure error. You can add all sorts of tasks, but a minimal pipeline needs only build and deploy tasks.

When setting up tasks, keep in mind that Azure DevOps includes an “assistant” that can make adding tasks much easier. To open the assistant, click “Show assistant,” which is on the upper right of the editing pane when you’re editing a YAML file.

For example, we can use this assistant to set up a NuGet task that will restore NuGet packages before building the solution. To do this, in the YAML editor we’ll move to the location right before the build task, open the assistant, scroll down the task list, and select the NuGet option:

We’ll then supply information that the assistant prompts us for, and we’ll click Add:

The assistant will generate code like the following, which we can modify, remove, etc., in the code editor:

If you are unfamiliar with writing YAML, using the assistant is a great way to get started adding tasks.

displayName:
The displayName keyword is used to specify the name that will be displayed for a task when the pipeline runs it. For example, in the following screen capture of a DevOps status window, the “Use .NET Core sdk 2.2x” label was not the original label for the task it represents. This label was added with a displayName setting.

inputs:
Inputs are the options that you can modify when adding a task. For example, the NuGet restore task we defined above takes required inputs for command, restoreSolution, and feedsToUse, which the assistant added for us.

Setting up deployment

The last part of our example pipeline will deploy the built app to an Azure web service. But before the pipeline can deploy, the Build stage needs to publish build artifacts for the pipeline. The following task lets the pipeline know where to get the files that are ready for deployment. This task and the other tasks we’re about to look at can also be created using the task assistant.

- task: PublishBuildArtifacts@1
      inputs:
        PathtoPublish: '$(Build.ArtifactStagingDirectory)'
        ArtifactName: 'drop'
        publishLocation: 'Container'

Then, in the Deploy stage, we need to download the build artifact to deploy:

- task: DownloadBuildArtifacts@1
      inputs:
        buildType: 'current'
        downloadType: 'single'
        downloadPath: '$(System.ArtifactsDirectory)'
        artifactName: 'drop'

Additionally, when deploying to an Azure app service, we must add a task to verify and authorize the Azure subscription and select the name of the app service the build will be deployed to. For example:

- task: AzureRmWebAppDeployment@4
      inputs:
        ConnectionType: 'AzureRM'
        azureSubscription: 'Visual Studio Enterprise Subscription – MPN(XXXXXXX)'
        appType: 'webApp'
        WebAppName: 'Some-Staging-Area'
        deployToSlotOrASE: true
        ResourceGroupName: 'Some-Staging-Area
        SlotName: 'staging'
        packageForLinux: '$(Build.ArtifactStagingDirectory)/**/*.zip'

In this example, packageForLinux is set to $(Build.ArtifactStagingDirectory)/**/*.zip because this is where the build artifact will be published in the last step of the Build stage. (The packageForLinux option is an alias for the  Package  keyword, and in the task assistant this is created using the “Package or folder” option.)

The completed pipeline

Now that we have a better sense of how YAML works and how we can use YAML to set up a pipeline, here’s a completed version of the pipeline we started above:

trigger:
- main

pool:
  vmImage: 'windows-latest'

variables:
  solution: '**/*.sln'
  buildPlatform: 'Any CPU'
  buildConfiguration: 'Release'

stages:
- stage: Build
  jobs:
  - job: Build
    steps:
    - task: NuGetCommand@2
      inputs:
        command: 'restore'
        restoreSolution: '**/*.sln'
        feedsToUse: 'select'

    - task: PowerShell@2
      displayName: Start CosmosDB Emulator
      inputs:
        targetType: 'inline'
        script: |
          Import-Module "$env:ProgramFiles\Azure Cosmos DB Emulator\PSModules\Microsoft.Azure.CosmosDB.Emulator"
            Start-CosmosDbEmulator

    - task: UseDotNet@2
      displayName: 'Use .NET Core sdk 2.2.x'
      inputs:
        version: 2.2.x

    - task: NuGetToolInstaller@1

    - task: NuGetCommand@2
      inputs:
        command: 'restore'
        restoreSolution: '$(solution)'
        feedsToUse: 'select'

    - task: VSBuild@1
      displayName: 'Build solution'
      inputs:
        solution: '$(solution)'
        msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(Build.ArtifactStagingDirectory)\\"'
        platform: '$(BuildPlatform)'
        configuration: '$(BuildConfiguration)'
        clean: true

    - task: VSTest@2
      displayName: 'Run Tests'
      inputs:
        testSelector: 'testAssemblies'
        testAssemblyVer2: |
          **\$(BuildConfiguration)\*Test*.dll
          **\$(BuildConfiguration)\**\*Test*.dll
          !**\*Microsoft.VisualStudio.TestPlatform*
          !**\obj\**
        searchFolder: '$(System.DefaultWorkingDirectory)'
        codeCoverageEnabled: true
        platform: '$(buildPlatform)'
        configuration: '$(buildConfiguration)'

    - task: PublishSymbols@1
      displayName: 'Publish symbols path'
      inputs:
        SearchPattern: '**\bin\**\*.pdb'
      continueOnError: true

    - task: PublishBuildArtifacts@1
      inputs:
        PathtoPublish: '$(Build.ArtifactStagingDirectory)'
        ArtifactName: 'drop'
        publishLocation: 'Container'

- stage: Deploy
  jobs:
  - job: Deploy
    steps:
    - task: DownloadBuildArtifacts@1
      inputs:
        buildType: 'current'
        downloadType: 'single'
        downloadPath: '$(System.ArtifactsDirectory)'
        artifactName: 'drop'
    - task: AzureRmWebAppDeployment@4
      inputs:
        ConnectionType: 'AzureRM'
        azureSubscription: 'Visual Studio Enterprise Subscription – MPN(XXXXXXX)'
        appType: 'webApp'
        WebAppName: 'Some-Staging-Area'
        deployToSlotOrASE: true
        ResourceGroupName: 'Some-Staging-Area
        SlotName: 'staging'
        packageForLinux: '$(Build.ArtifactStagingDirectory)/**/*.zip'

When this pipeline is triggered, it will build, run tests, and deploy:

As you can see, YAML gives you a great deal of control for building, testing, and deploying projects. We’ve looked at a simple example of what can be done with YAML, but there are a plenty of complexities that can come into play, and there are other neat tricks that are possible with YAML pipelines. Hopefully it’s apparent that with the right tooling CI/CD isn’t a pipe dream. It is achievable, and it can put you on the path to greater efficiency and faster delivery of your products.

The post YAML Your Way to Rapid Deployment appeared first on Synergex.

The Select class in Synergy DBL allows developers to access data from Synergy DBMS files using syntax and concepts that are closer to SQL than to the traditional file-based approach. Since its introduction, the Select class has grown to include more SQL-like features. With Synergy/DE 12, we’ve taken another step in that direction by introducing a new class that can be used with Select: GroupBy. This feature was initially introduced in Synergy/DE 12.1 but has been refined in the recent 12.2.1.1003 feature release.

The GroupBy class implements some of the functionality found in the GROUP BY and DISTINCT keywords in SQL. GroupBy enables you to filter results from a Select query so that similar records are treated as groups instead of individual records. It does this by letting you specify one or more fields, which it will use to sort the result set. Any set of records with the same value in those fields will be treated as a single group, and only one record from the group will be returned as a representative of the whole.

Basic Syntax The Synergy GroupBy class is derived from the OrderBy class, but the syntax is a bit different. Instead of using the Ascending and Descending methods, you can use a constructor and specify one or more fields to group by, similar to the Sparse class. For example

new GroupBy(firstname)
new GroupBy(lastname, firstname)

If you want to specify multiple fields, you can combine multiple GroupBy objects with the .AND. or && operators or just list multiple fields in the same call to Ascending() or Descending(). You can even combine both forms. For example:

grpobj1 = new GroupBy(firstname,lastname)
grpobj2 = new GroupBy(middleinitial)
grpobj3 = grpobj1.and.grpobj2
grpobj4 = new GroupBy(firstname,lastname) && (new GroupBy(middleinitial))

In any case, the leftmost field has the highest priority, and the rightmost field has the lowest priority. For grpobj3 and grpobj4 in the example above, records would be sorted first by firstname, then by lastname, and finally by middleinitial.

Ordering is an inherent concept in GroupBy: the returned groups will always be sorted by the specified fields in ascending order.

The constructor generates a GroupBy object, which can either be used inline in a Select constructor or assigned to a variable for later use. For instance:

selobj = new select(sprsobj, fromobj, new GroupBy(firstname,lastname))

or

         grpobj = new GroupBy(firstname,lastname)
selobj = new select(sprsobj, fromobj, grpobj)

Whenever you use a GroupBy object in a Select query, you must also use a Sparse object. The reasons for this are described in the Sparse section below. For now, the examples will use “Sparse(“*”)”, which selects all fields. For example:

selobj = new select(new Sparse("*"), fromobj, new GroupBy(lastname))

Basic Example

To see this in action, consider the following example:

structure employee
    key,           d5
    ,              a1
    firstname,     a15
    ,              a1
    lastname,      a15
    ,              a1
    department,    a15
    ,              a1
    salary,        d7
endstructure

import Synergex.SynergyDE.Select
.main
record
    sprsobj, @Sparse
    fromobj, @From
    selobj,  @Select
    rec1,            employee
record
    count,         i4
    salarysum,     d8
    salaryavg,     d7

.define FILE "DAT:employees.ism"
.proc
    xcall flags(7000000,1)
    open(1, o, "TT:")
    xcall isamc(FILE, ^size(rec1), 1,
&               "START=1, LENGTH=5, NAME=""Key1"", DUPS, ASCEND, ATEND")
        open(2, u:I, FILE)
        store(2, "00001 Andrew          McDonald        Development     0025010")
        store(2, "00002 Bob             Brown           HR              0030020")
        store(2, "00003 Beth            Jones           HR              0035030")
        store(2, "00004 Cathy           Brown           Sales           0040040")
        store(2, "00005 David           SMITH           HR              0045050")
        store(2, "00006 Fred            Smith           QA              0050060")
        store(2, "00007 George          Cartwright      Shipping        0055070")
        store(2, "00008 Henry           Smith           Development     0060080")
        store(2, "00009 Jennifer        Jones           QA              0065090")
        store(2, "00010 James           Mcdonald        Development     0070000")
        store(2, "00011 John            Cartwright      Development     0075010")
        store(2, "00012 Laura           Jones           QA              0080020")
        store(2, "00013 Matthew         Cartwright      Shipping        0085030")
        store(2, "00014 Phillip         Brown           Sales           0090040")
        store(2, "00015 Rose            Jones           Sales           0095050")
        close 2
        open(2, I:I, FILE)
        fromobj = new from(2, rec1)
        writes(1, "")
        writes(1, "OrderBy Department:")
        writes(1, "KEY   FIRSTNAME       LASTNAME        DEPARTMENT      SALARY")
        selobj = new select(fromobj, OrderBy.Ascending(rec1.department))
        foreach rec1 in selobj
        begin
          writes(1,rec1)
        end
        writes(1, "")
        writes(1, "GroupBy Department:")
        writes(1, "KEY   FIRSTNAME       LASTNAME        DEPARTMENT      SALARY")
        sprsobj = new Sparse("*")
        selobj = new select(sprsobj, fromobj, new GroupBy(rec1.department))
        foreach rec1 in selobj
        begin
          writes(1,rec1)
        end
        writes(1, "")
        close 2
        close 1
        stop
.end

This program creates an ISAM file containing an example table of employees. It runs a Select query with OrderBy to retrieve the full table sorted by department:

OrderBy Department:
KEY   FIRSTNAME       LASTNAME        DEPARTMENT      SALARY
00001 Andrew          McDonald        Development     0025010
00008 Henry           Smith           Development     0060080
00010 James           McDonald        Development     0070000
00011 John            Cartwright      Development     0075010
00002 Bob             Brown           HR              0030020
00003 Beth            Jones           HR              0035030
00005 David           Smith           HR              0045050
00006 Fred            Smith           QA              0050060
00009 Jennifer        Jones           QA              0065090
00012 Laura           Jones           QA              0080020
00004 Cathy           Brown           Sales           0040040
00014 Phillip         Brown           Sales           0090040
00015 Rose            Jones           Sales           0095050
00007 George          Cartwright      Shipping        0055070
00013 Matthew         Cartwright      Shipping        0085030

It then runs the same query again, using GroupBy instead of OrderBy:

GroupBy Department:
KEY   FIRSTNAME       LASTNAME        DEPARTMENT      SALARY
00001 Andrew          McDonald        Development     0025010
00002 Bob             Brown           HR              0030020
00006 Fred            Smith           QA              0050060
00004 Cathy           Brown           Sales           0040040
00007 George          Cartwright      Shipping        0055070

In the second query, all of the records with a certain value for department are treated as a single group, and only one record is returned for each group.

Suppose we changed the OrderBy and GroupBy objects to use both rec1.department and rec1.lastname fields instead of just rec1.department:

selobj = new select(fromobj, OrderBy.Ascending(rec1.department, rec1.lastname))
selobj = new select(sprsobj, fromobj, new GroupBy(rec1.department, rec1.lastname))

The OrderBy query would return the same results as before, but in a slightly different order:

OrderBy Department and Lastname:
KEY   FIRSTNAME       LASTNAME        DEPARTMENT      SALARY
00011 John            Cartwright      Development     0075010
00001 Andrew          McDonald        Development     0025010
00010 James           Mcdonald        Development     0070000
00008 Henry           Smith           Development     0060080
00002 Bob             Brown           HR              0030020
00003 Beth            Jones           HR              0035030
00005 David           SMITH           HR              0045050
00009 Jennifer        Jones           QA              0065090
00012 Laura           Jones           QA              0080020
00006 Fred            Smith           QA              0050060
00004 Cathy           Brown           Sales           0040040
00014 Phillip         Brown           Sales           0090040
00015 Rose            Jones           Sales           0095050
00007 George          Cartwright      Shipping        0055070
00013 Matthew         Cartwright      Shipping        0085030

But the GroupBy query would return additional records that weren’t present in the original:

GroupBy Department and Lastname:
KEY   FIRSTNAME       LASTNAME        DEPARTMENT      SALARY
00011 John            Cartwright      Development     0075010
00001 Andrew          McDonald        Development     0025010
00010 James           Mcdonald        Development     0070000
00008 Henry           Smith           Development     0060080
00002 Bob             Brown           HR              0030020
00003 Beth            Jones           HR              0035030
00005 David           SMITH           HR              0045050
00009 Jennifer        Jones           QA              0065090
00006 Fred            Smith           QA              0050060
00004 Cathy           Brown           Sales           0040040
00015 Rose            Jones           Sales           0095050
00007 George          Cartwright      Shipping        0055070

This is because the GroupBy query is now using both fields to determine groups. Records in the new query will only be considered as part of the same group if they have identical values for both department and lastname, instead of just department. If multiple employee records in the same department have the same lastname value, they’ll still be part of the same group, but different last names will result in separate groups.

For the sake of simplicity, the examples here select all records from the data file. But GroupBy will also work with more complicated queries that include a Where expression.

Case Sensitivity

You may have noticed from the previous example that there appear to be two groups with a department value of “Development” and a lastname value of “McDonald”. This is because the default behavior for OrderBy and GroupBy is to use case-sensitive comparisons when sorting records. So “McDonald” and “Mcdonald” are distinct values and result in separate groups in GroupBy. This is easy to see if we group by lastname only:

GroupBy Lastname:
KEY   FIRSTNAME       LASTNAME        DEPARTMENT      SALARY
00002 Bob             Brown           HR              0030020
00007 George          Cartwright      Shipping        0055070
00003 Beth            Jones           HR              0035030
00001 Andrew          McDonald        Development     0025010
00010 James           Mcdonald        Development     0070000
00005 David           SMITH           HR              0045050
00006 Fred            Smith           QA              0050060

If you want to ignore differences in case, you can use the NoCaseGroupBy class instead of the regular GroupBy:

NoCaseGroupBy Lastname:
KEY   FIRSTNAME       LASTNAME        DEPARTMENT      SALARY\
00002 Bob             Brown           HR              0030020
00007 George          Cartwright      Shipping        0055070
00003 Beth            Jones           HR              0035030
00001 Andrew          McDonald        Development     0025010
00005 David           SMITH           HR              0045050

We’ve also added the same functionality to OrderBy in the new NoCaseOrderBy class. Note that NoCaseGroupBy and NoCaseOrderBy are only intended to be used with alpha fields and may cause problems if used with any other data type. If you want to use case-insensitive comparisons for some fields and not others, you can create multiple objects and combine them. For example:

new NoCaseGroupBy(alpha_field) && (new GroupBy(int_field))

Sparse

A Select query using GroupBy will return a record from the data file to represent each group as a whole. Because this is a regular record, we can expect it to contain data in all of its fields, not just the fields used to determine the group. But these fields will probably be meaningless for use in representing the group. We can see this by going back to the “GroupBy(rec1.department)” example:

GroupBy Department:
KEY   FIRSTNAME       LASTNAME        DEPARTMENT      SALARY
00001 Andrew          McDonald        Development     0025010
00002 Bob             Brown           HR              0030020
00006 Fred            Smith           QA              0050060
00004 Cathy           Brown           Sales           0040040
00007 George          Cartwright      Shipping        0055070

The only meaningful field in that data is the one used to determine the groups: department. The fact that the employee record returned for the Development department has a name of “Andrew McDonald” doesn’t necessarily tell us anything about the Development department. And the fact that the records for the HR and Sales departments both have a lastname value of “Brown” doesn’t necessarily indicate any similarity between those departments.

It would be helpful to limit the returned data to fields that actually matter. In a SQL query, this could be done by writing a query to only return the fields you care about. In Synergy’s Select class, the fields used by the returned data are determined in advance by the record definition. But you can use the Sparse class to specify the fields that you want returned, leaving all other fields empty. For example:

selobj = new select(new Sparse(rec1.department), fromobj,
&                   new GroupBy (rec1.department))

which results in

GroupBy Department (Sparse):
KEY   FIRSTNAME       LASTNAME        DEPARTMENT      SALARY
                                      Development
                                      HR
                                      QA
                                      Sales
                                      Shipping

The primary use case for Sparse is to improve performance when accessing data over xfServer by reducing the amount of data sent over the network. Our documentation warns against using Sparse when you aren’t also using xfServer, as it may not improve performance. But in the case of GroupBy, Sparse can still be useful regardless of any performance impact.

Additionally, because GroupBy now uses a static cursor to sort and return results, Sparse may improve performance even for local files. A static cursor reads all matching records from a data file into memory before processing (e.g., sorting and grouping) and returning them. Using Sparse to reduce the number of returned fields will reduce the amount of data that needs to be kept in memory, which can reduce memory usage and improve performance.

For these reasons, a Sparse object is required for all queries that use a static cursor, such as all GroupBy queries. We strongly recommend using Sparse to limit a GroupBy query to fields that are actually meaningful, but it is still possible to include additional fields, or even to specify “Sparse(“*”)” and include all fields regardless of relevance.

Aggregate Values

While the non-group fields of any given GroupBy record may not be meaningful individually, it may be helpful to use these fields to describe a group as a whole. For instance, it may not be useful to see the salary of the employee record returned for the QA department, because the record might not be representative of the department. But if you could see the average salary for everyone in the department, or the total amount of money the department spends on payroll, that information could be useful.

This brings us to a primary use case of GROUP BY in SQL: aggregate values. In addition to the actual fields from a table, SQL allows you to select aggregate functions to be performed for all fields in a group, such as Sum, Count, Average, Minimum, and Maximum.

At present, the Synergy GroupBy class doesn’t have aggregate functions built in. But it’s possible to calculate aggregate values in code using multiple Select objects. The first Select object can use GroupBy to retrieve distinct values for specific fields. Then a second Select object can use those values to drive a Where expression and iterate over all records in that group. The code can use those records to calculate any aggregate values.

Advanced Example

To see an example of aggregate values, you can add this code after the existing Select objects in the test program:

writes(1, "GroupBy Department with aggregate values:")
            writes(1, "DEPARTMENT         COUNT     AVG(SALARY)  SUM(SALARY)")
            selobj = new select(new Sparse(rec1.department), fromobj,
&                               new GroupBy(rec1.department))
            foreach rec1 in selobj
            begin
                data selobj2, @Select
                data rec2, employee
                open(3, I:I, FILE)
                count = 0
                salarysum = 0
                salaryavg = 0
                selobj2 = new Select(new From(3,rec2),
&                                    (Where)(rec2.department == rec1.department))
                foreach rec2 in selobj2
                  begin
                      count += 1
                      salarysum += rec2.salary
                  end
                salaryavg = salarysum/count
                writes(1,rec1.department + "    " + %string(count,"XX") +
&                      "    " + %string(salaryavg,"$$$,$$$,$$$") +
&                      "    " + %string(salarysum,"$$,$$$,$$$"))
                close 3
            end

After running this program, you should get output like this:

GroupBy Department with aggregate values:
DEPARTMENT         COUNT     AVG(SALARY)  SUM(SALARY)
Development        04        $57,525      $230,100
HR                 03        $36,700      $110,100
QA                 03        $65,056      $195,170
Sales              03        $75,043      $225,130
Shipping           02        $70,050      $140,100

This code uses GroupBy to return individual records for each department. Within the outer FOREACH loop, another Select object retrieves all records that match each group and keeps a running total of the number of records and the salary for all records in the department. After it’s done reading the records, the code calculates the average salary for the department and displays the data to the screen.

A downside to this method is that there’s no built-in way to sort groups based on an aggregate value. The order of groups is based on the field used in the GroupBy, and any ordering based on aggregate values would need to be done after the initial Select is finished and all aggregate values are calculated.

Limitations

At the time of this writing, the current syntax for GroupBy (introduced in 12.2.1.1003) can only be used when targeting the latest runtime. If you’re building a Synergy program from the command line, this means you must include an option like “-qrntcompat=120301” in the compiler and linker commands. If you’re building from Visual Studio, you can select a Target Synergy runtime of “Latest” in the Build properties for the project. (The original syntax introduced in 12.1 that uses Ascending() and Descending() methods, similar to the OrderBy class, can be used when targeting the default 120101 runtime level, but that syntax is no longer recommended.)

GroupBy requires compiler and runtime features that aren’t present in earlier versions of Synergy. You won’t be able to compile code that uses GroupBy if you target an earlier Synergy version. (In other words, if you use the -qrntcompat compiler and linker option, you can’t specify a version number prior to 12010100.) Likewise, if you’re accessing data over xfServer, the server must be running at least version 12 for GroupBy to work. By default, an xfServer client of version 12.2.1.1003 or later can only connect to a server of version 12.2.1.1003 or later, although you can use the SRV_COMPAT environment variable to override this and connect to a server with an earlier version.

At present, GroupBy is not supported with a JoinSelect. Queries using GroupBy must operate on a single file only. Likewise, GroupBy and OrderBy are not supported on the same query, although results returned by GroupBy will be sorted in ascending order.

GroupBy uses a static cursor and so operates on a snapshot of the data file at the time when the enumerator is created. As a result, it’s possible for the underlying data to change between the time when the Select statement starts working and the time when it retrieves any given record. This shouldn’t change the order of any groups, but it could result in new groups being skipped if records belonging to that group are only added after the Select has started working.

Because each result returned by a GroupBy query represents a group of records instead of a single record in a file, several Select features that require individual records won’t work. For instance, the RestrictedAlphaEnumerator methods CurrentRFA and DeleteCurrent are incompatible with GroupBy. A GroupBy result will have no change tracking information, so the GetCTInfo property is also incompatible. Record locking is incompatible with a GroupBy query, and a GroupBy query won’t execute if locks are enabled in the From object, so the IsLocked property is incompatible as well. The SparseUpdate method is also incompatible, because it can’t be used on a query with a Sparse object, and Sparse objects are required when using GroupBy.

In some circumstances (e.g., if it’s sorting by a single key that’s already in the correct order in the ISAM file), OrderBy can optimize a query by bypassing the process of sorting records and returning them in the correct order directly from the file. GroupBy doesn’t support this but will always sort the result set regardless of what fields are specified. This may result in slower performance than would otherwise be possible for some queries.

As mentioned above, aggregate functions are not currently supported with GroupBy.

Future Plans

While future plans have not been finalized, we’re considering addressing many of the above limitations in future releases of Synergy/DE. One of the biggest improvements we’d like to make in the near future is to support GroupBy with JoinSelect, similar to the support we recently added to use OrderBy with JoinSelect. We’d also like to support OrderBy and GroupBy on the same queries.

In the longer term, we’d like to support aggregate functions with GroupBy (Sum, Count, Average, Minimum, and Maximum), bringing feature parity closer to SQL. Ideally, this feature would allow arbitrary mapping of grouped and aggregated fields from a source record into a destination record, with arbitrary sorting of the results. This mapping could also be useful in other contexts besides GroupBy.

Another potential change is to optimize GroupBy, perhaps by returning records directly from memory after sorting them, instead of reading records from the ISAM file when retrieving them. We might also have GroupBy imply Sparse, so results aren’t cluttered by meaningless records and performance over xfServer is improved, without requiring an explicit definition of a Sparse object.

These changes will likely require new versions of the compiler, runtime, and xfServer, so backwards compatibility is one limitation we wouldn’t be addressing.

Conclusion

I hope this article has given you a good introduction to the GroupBy class. You should have some idea of how the class can be used, both theoretically and with practical code examples. You should also have a basic understand of the differences between the Synergy GroupBy class and SQL, as well as the current limitations of the class, but also some insight into how the class may develop in the future.

The post Select.GroupBy: What’s New and What’s to Come appeared first on Synergex.