Subtopic: Software development / 3

Company and industry news, featured projects, open source code, tech tips, and more.

Automate the installation of all your Mac apps using Homebrew and Mas

Michael Argentini Avatar
Michael ArgentiniWednesday, July 10, 2024

On macOS it's pretty easy to automate the installation of all your apps, including Mac App Store apps, for those times when you get a new Mac or wipe your current one. As a software developer I find this capability indispensable, as would any professional or power user.

All you need to do is install Homebrew and then use it to install mas (which is an acronym for Mac App Store). Once they are installed, you can install all your software using a convenient Bash script. Homebrew will be used to install non-store apps, and mas will handle installing the Mac App Store apps.

Note: only Mac App Store apps you have already installed previously can be installed with mas. You cannot install new apps you have never installed from the store.

Why do this?

The most obvious reason to script out your software installations is that it greatly reduces the time to set up a new Mac, as well as ensure that you don't forget to install one or more apps. It also provides a way to update all the apps at once via the brew upgrade command. And it also provides a way to update apps that don't have their own update feature.

Apps installed with mas will be updated normally by the Mac App Store.

How does this work?

In order to use this process you need to know the names of the Homebrew formulae/casks for each application, and you also need to know the IDs of the Mac App Store apps for mas. Fortunately this is super easy.

First, Homebrew has a tool for finding software available in their service at https://formulae.brew.sh/. Simply use this to find your apps and make sure you're installing the right ones. Those listed as “casks” are GUI Mac apps (normal apps you don't run from the Terminal). Ones listed as “formulae” are typically command line tools run from the Terminal or services without an interface.

Second, for Mac App Store apps you simply use mas to list what's currently on your computer from the Mac App Store.

mas list

This will give you a list of currently installed apps from the Mac App Store, with their IDs:

1569813296  1Password for Safari      (2.10.0)
975937182   Fantastical               (3.7.12)
409183694   Keynote                   (13.0)
etc.

You can also search for Mac App Store apps by name using the mas search command:

mas search Xcode

This will show a similar result for matches. You can even install all search results with a single “lucky” command. See the mas help for these and other options.

Script example

Here's an example of a Bash script to get you started. I keep a similar script updated as I use new apps or stop using others. Then I'm ready to go when I have to set up a new or wiped Mac.

# Install Homebrew

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

# App Store Automation

brew install mas

# Install App Store Apps

mas install 409183694   # Keynote
mas install 409201541   # Pages
mas install 409203825   # Numbers

# Install Non-Store Apps

brew install --cask firefox
brew install --cask knockknock
# etc.

You can name the Bash script something like install-software.sh and execute it in a Terminal like this:

zsh install-software.sh

The first time you use the script will absolutely justify the time spent writing it. The second time you run it you will thank your past self for being so smart 😉.

Want to know more?

There's usually more to the story so if you have questions or comments about this post let us know!

Do you need a new software development partner for an upcoming project? We would love to work with you! From websites and mobile apps to cloud services and custom software, we can help!

SqlPkg for Microsoft SqlPackage

Michael Argentini Avatar
Michael ArgentiniMonday, July 1, 2024

SqlPkg is a 64-bit .NET command line (CLI) wrapper for the Microsoft SqlPackage CLI tool with the goal of making common backup and restore operations easier and more powerful. It does this through new Backup and Restore actions that provide additional features like the exclusion of specific table data in backups and destination prep prior to restore.

Visit the repository to see how you can install this tool to begin using it right away.

New action modes:

/Action:backup
This mode is equivalent to Action:Export to create a .bacpac file, with the following differences.

  • Specify one or more /p:ExcludeTableData= properties to exclude specific table data from the bacpac file. The table name format is the same as the /p:TableData= property.
  • /SourceTrustServerCertificate: defaults to true.
  • /SourceTimeout: defaults to 30.
  • /CommandTimeout: defaults to 120.
  • /p:VerifyExtraction= defaults to false.
  • Destination file paths will be created if they do not exist.

/Action:restore
This mode is equivalent to Action:Import to restore a .bacpac file, with the following differences.

  • The destination database will be purged of all user objects (tables, views, etc.) before the restoration.
  • If the destination database doesn't exist it will be created.
  • /TargetTrustServerCertificate: defaults to true.
  • /TargetTimeout: defaults to 30.
  • /CommandTimeout: defaults to 120.
  • Destination file paths will be created if they do not exist.

/Action:backup-all
This mode will back up all user databases on a server.

  • Provide a source connection to the master database.
  • Provide a target file path ending with 'master.bacpac'. The path will be used as the destination for each database backup file, ignoring 'master.bacpac'.
  • Optionally provide a log file path ending with 'master.log'. The path will be used as the destination for each database backup log file, ignoring 'master.log'.
  • Accepts all arguments that the Backup action mode accepts.

/Action:restore-all
This mode will restore all *.bacpac files in a given path to databases with the same names as the filenames.

  • Provide a source file path to 'master.bacpac' in the location of the bacpac files. The path will be used as the source location for each database backup file to restore, ignoring 'master.bacpac'.
  • Provide a target connection to the master database.
  • Optionally provide a log file path ending with 'master.log'. The path will be used as the destination for each database backup log file, ignoring 'master.log'.
  • Accepts all arguments that the Restore action mode accepts.

When not using SqlPkg special action modes, the entire argument list is simply piped to SqlPackage and will run normally. So you can use sqlpkg everywhere SqlPackage is used.

Installation

1. Install Microsoft .NET

SqlPkg requires that you already have the .NET 8.0 runtime installed, which you can get at https://dotnet.microsoft.com/en-us/download.

Because SqlPkg uses Microsoft SqlPackage, you will also need to install the .NET 6.0 runtime as well as SqlPackage.

dotnet tool install -g microsoft.sqlpackage

2. Install SqlPkg

Run the following command in your command line interface (e.g. cmd, PowerShell, Terminal, bash, etc.):

dotnet tool install --global fynydd.sqlpkg

Later you can update SqlPkg with the following command:

dotnet tool update --global fynydd.sqlpkg

Uninstall

If you need to completely uninstall SqlPkg, use the command below:

dotnet tool uninstall --global fynydd.sqlpkg
Screenshots

Want to know more?

There's usually more to the story so if you have questions or comments about this post let us know!

Do you need a new software development partner for an upcoming project? We would love to work with you! From websites and mobile apps to cloud services and custom software, we can help!

Fdeploy for web app deployments

Michael Argentini Avatar
Michael ArgentiniSaturday, June 1, 2024

The Fdeploy project is a command line interface (CLI) application that can use simple YAML config files in your ASP.NET web projects to define deployment to one or more remote environments over SMB, like over a VPN connection using a network file share.

It can be configured to clean and purge the project, build, and publish, and even add files and folders to the published output prior to deployment.It can then deploy with various rules like "path ignore", "always update path", "clean orphaned files and folders", and more. You can even define content that can be deployed without taking the web app offline. And when it does, it uses an app offline file that you can also customize in the YAML settings.

Fdeploy also has robust retry support. When files are in-use on the remote server it will retry one or more times and wait a specified number of seconds between attempts.

Visit the repository to see how you can install this tool to begin using it right away.

Installation

1. Install Microsoft .NET

Fdeploy requires that you already have the .NET 8.0 runtime installed, which you can get at https://dotnet.microsoft.com/en-us/download.

2. Install Fdeploy

Run the following command in your command line interface (e.g. cmd, PowerShell, Terminal, bash, etc.):

dotnet tool install --global fynydd.fdeploy

Later you can update Fdeploy with the following command:

dotnet tool update --global fynydd.fdeploy

Uninstall

If you need to completely uninstall Fdeploy, use the command below:

dotnet tool uninstall --global fynydd.fdeploy
Screenshots

Want to know more?

There's usually more to the story so if you have questions or comments about this post let us know!

Do you need a new software development partner for an upcoming project? We would love to work with you! From websites and mobile apps to cloud services and custom software, we can help!

Namecheap DNS

Michael Argentini Avatar
Michael ArgentiniWednesday, May 15, 2024

The NameCheap project builds a command line interface (CLI) application that can add and remove text records using the NameCheap API.

It was originally built to allow for creating wildcard TLS certificates using win-acme (Let's Encrypt) on an IIS server. Creating wildcard certificates with win-acme requires DNS host validation. This application can be used with the win-acme script feature to allow it to communicate with the NameCheap API and create/delete TXT records that will validate domain ownership.

This tool does not support the complete NameCheap API. But it does handle the challenging task of adding and removing text records. Why is this challenging? The NameCheap API does not have functions to add or remove individual records, so the entire set of records must be downloaded, modified, and sent back.

How to install

Download the project and publish it from the root project folder as below.

dotnet publish Fynydd.NameCheap/Fynydd.NameCheap.csproj -o publish -p:PublishSingleFile=true -c Release -r win-x64 --self-contained

In the publish folder, edit the appsettings.json file and supply your own values.

{
    "NameCheap": {

        "ApiKey": "{your namecheap API key}",
        "UserName": "{your namecheap username}",
        "ApiUserName": "{your namecheap API username}",
        "ClientIP": "{a whitelisted IPv4 address}"
    }
}

Note:

  • You can enable the NameCheap API and get a key on their website.
  • UserName and ApiUserName are usually the same value, and it is usually the user name you use to sign in to NameCheap.
  • ClientIP is a whitelisted IP address allowed to connect to the API. These whitelisted addresses can be added to NameCheap when/where you enable the API on their website. Note: API calls will check your current WAN IP with the one you provide in the settings. So they need to match.

Once the appsettings.json file is modified, put the contents of the publish folder on your server and you should be able to use the executable with win-acme or any other tool by calling it with a fully qualified path.

Usage

The command line is in the format below:

Fynydd.NameCheap.exe [create|delete] [hostname] [name] [value]

Some examples include:

Fynydd.NameCheap.exe create example.com * testrecord=yaddayadda
Fynydd.NameCheap.exe create example.com my.api mykey=yaddayadda
Fynydd.NameCheap.exe create example.com my.txt "val1=yadda; val2=yadda"

So in win-acme you would set your create script arguments to this:

create {ZoneName} {NodeName} {Token}

Likewise, your delete script arguments would be:

delete {ZoneName} {NodeName} {Token}

macOS and Linux

The tool can be used on Linux or macOS as well. If the published executable doesn't run on macOS you may need to manually sign the published application using something like this:

cd publish
codesign -s - Fynydd.NameCheap

Want to know more?

There's usually more to the story so if you have questions or comments about this post let us know!

Do you need a new software development partner for an upcoming project? We would love to work with you! From websites and mobile apps to cloud services and custom software, we can help!

Benford's Law

Michael Argentini Avatar
Michael ArgentiniFriday, March 15, 2024

The Benford project provides a framework for using Benford's Law to determine if a given random data set has been modified.

Benford’s Law, also known as the Law of First Digits, the Phenomenon of Significant Digits, or the Law of Anomalous Numbers is the finding that the first digits (or numerals to be exact) of the numbers found in series of records of the most varied sources do not display a uniform distribution, but rather are arranged in such a way that the digit “1” is the most frequent, followed by “2”, “3”, and so in a successively decreasing manner down to “9”. The expected distributions can be seen in the screenshots below.

Essentially, when looking at a significantly large set of numbers, this law can tell you if the distributions of first digits follows Benford's Law. Deviations of significance should indicate data tampering. This can be applied in all kinds of use cases, from social likes and follower counts, to regional vote counts, financial information, and more.

This project has some sample images and voting data that you can use to see how Benford's Law applies to them.

Screenshots

Want to know more?

There's usually more to the story so if you have questions or comments about this post let us know!

Do you need a new software development partner for an upcoming project? We would love to work with you! From websites and mobile apps to cloud services and custom software, we can help!

DataStore

Michael Argentini Avatar
Michael ArgentiniThursday, February 15, 2024

The DataStore project is a high performance JSON object store (ORM) for SQL Server.

DataStore uses and automatically creates and manages a pre-defined SQL Server data structure that can coexist with existing database objects. All database operations are performed with the DataStore helper class.

Your models are stored in the database as JSON text so you can have most any kind of object structure, provided your models inherit from DsObject.

Basic example

Instantiating DataStore with settings is non-destructive. Any existing DataStore tables are left untouched. Methods to delete all or unused schema objects are provided for those edge cases.

Models

Instantiate DataStore with a settings object and database schema will be created for all classes that inherit from DsObject. The following attributes can be used in your classes:

  • DsNoDatabaseTable prevents DataStore from creating a table for the class.
  • DsUseLineageFeatures enables lineage features for that table; add to the class itself.
  • DsSerializerContext(typeof(...)) to provide a de/serialization speed boost by using source generator JsonSerializationContext classes for each table; add to the class itself.
  • DsIndexedColumn generates a SQL computed column with index for faster queries on that data; add to properties and fields.
  • DsIndexedColumn("Food","Email") generates indexed SQL computed columns for faster queries on the dictionary key names specified; add to Dictionary properties and fields.
[DsUseLineageFeatures]
[DsSerializerContext(typeof(UserJsonSerializerContext))]
public class User: DsObject
{
    [DsIndexedColumn]
    public string Firstname { get; set; }
    
    [DsIndexedColumn]
    public int Age { get; set; }
    
    public List<Permissions> Permissions { get; set; }
    
    [DsIndexedColumn("Food", "Color")]
    public Dictionary<string, string> Favorites { get; set; } = new();
}

[JsonSerializable(typeof(User))]
[JsonSourceGenerationOptions(WriteIndented = false)]
internal partial class UserJsonSerializerContext : JsonSerializerContext
{ }

Construction

You can create a DataStore instance anywhere in your code:

var dataStore = new DataStore(new DataStoreSettings {
    SqlConnectionString = sqlConnectionString,
    UseIndexedColumns = true
});

You can also use DataStore as a singleton service:

services.AddSingleton<DataStore>((factory) => new DataStore(new DataStoreSettings {
    SqlConnectionString = sqlConnectionString,
    UseIndexedColumns = true
}));

Create and save objects

Creating and saving a DataStore object is simple:

var user = new User
{
    FirstName = "Michael",
    LastName = "Fynydd",
    Age = 50,
    Permissions = new List<Permission>
    {
        new() { Role = "user" },
        new() { Role = "admin" },
        // etc.
    }
};

await dataStore.SaveAsync(user);

The saved object is updated with any changes, like lineage and depth information, creation or last update date, etc. And you can provide a list of objects to save them all in one call.

Read objects

Querying the database for objects is simple too. In any read calls you can specify a DsQuery object with a fluent-style pattern for building your query. In the query you can specify property names as strings with dot notation:

var users = await dataStore.GetManyAsync<User>(
    page: 1,
    perPage: 50,
    new DsQuery()
        .StringProp("LastName").EqualTo("Fynydd")
        .AND()
        .StringProp("Permissions.Role").EqualTo("admin")
        .AND()
        .GroupBegin()
            .NumberProp<int>("Age").EqualTo(50)
            .OR()
            .NumberProp<int>("Age").EqualTo(51)
        .GroupEnd(),
    new DsOrderBy()
        .Prop<int>("Age").Ascending()
);

Or you can use the model structure to specify names, and make code refactoring easier:

var users = await dataStore.GetManyAsync<User>(
    page: 1,
    perPage: 50,
    new DsQuery()
        .StringProp<User>(u => u.LastName).EqualTo("Fynydd")
        .AND()
        .StringProp<User, Role>(u => u.Permissions, r => r.Role).EqualTo("admin")
        .AND()
        .GroupBegin()
            .NumberProp<User,int>(u => u.Age).EqualTo(50)
            .OR()
            .NumberProp<User,int>(u => u.Age).EqualTo(51)
        .GroupEnd(),
    new DsOrderBy()
        .Prop<User>(o => o.Age).Ascending()
);

Dynamic property access

If you need to access object properties without knowing the object type, DsObject exposes JSON features that allow you to access property values using standard JSON path syntax:

var users = await dataStore.GetManyAsync<User>(
    page: 1,
    perPage: 50
);

foreach (DsObject dso in users)
{
    dso.Serialize(dataStore);

    var lastName = dso.Value<string>("$.LastName");
    var roles = dso.Values(typeof(string), "$.Permissions.Role");

    // etc.
}

Remember: these JSON features are read-only. If you change a property value in the DsObject you will need to call Serialize() again to update the JSON representation.

Screenshots

Want to know more?

There's usually more to the story so if you have questions or comments about this post let us know!

Do you need a new software development partner for an upcoming project? We would love to work with you! From websites and mobile apps to cloud services and custom software, we can help!

Upgrade Windows 10 without a TPM chip

Michael Argentini Avatar
Michael ArgentiniSunday, November 12, 2023

Microsoft (in)famously requires that your PC have a special security chip (TPM) in order to install Windows 11. But the requirement is wholly unnecessary to maintain a secure installation. And it requires many people to buy new PCs, arguing that this is likely the primary reason Microsoft implemented the requirement.

Patching the Windows install to allow it to run on existing hardware without a TPM chip has been a game of cat and mouse. And until now it had again been thwarted by Microsoft.

Recently Microsoft has patched the Windows 11 ‘product server’ trick for TPM check bypass, but the bypass still works with setupprep.exe. This bypass will upgrade Windows 10 clients to Windows 11 without requiring a TPM.

setupprep.exe /product server

Prioritizing software features

Michael Argentini Avatar
Michael ArgentiniFriday, October 13, 2023

Being a pragmatist when it comes to software development, I generally prefer simple solutions whenever possible. So when clients are faced with the challenge of determining which features to add to their products, as a starting point I usually recommend looking at user value versus cost to implement.

Given a progressive value scale from a “cup of coffee” at the low end and a “house” at the high end, we can easily see where the best choices lie.

Looking at the first row in this example, if a user values a feature at “house” and your cost to implement is a “cup of coffee” this feature addition becomes a no-brainer.

Likewise if a user values a feature at a “cup of coffee” but it will cost a “house” to build, it's certainly a feature to avoid implementing.

There may be some diamonds in the center blue area, but more information is usually required before we make those decisions.

Want to know more?

There's usually more to the story so if you have questions or comments about this post let us know!

Do you need a new software development partner for an upcoming project? We would love to work with you! From websites and mobile apps to cloud services and custom software, we can help!

How to copy a Git repo to another location using rsync on macOS

Michael Argentini Avatar
Michael ArgentiniWednesday, June 14, 2023

Here's a Mac pro tip: When copying a git repository directory to your network file server using rsync, try this:

rsync -brtlvP --chmod=Fu+w --delete-before Code/MyProject/ /Volumes/MyServer/Code/MyProject/

This will copy the files with timestamps but no other attributes, and will also give the file owner (you) full access rights. This ensures that you (or even Windows users with access rights) can overwrite the directory contents later because local read-only files (e.g. in the .git directory) won't be read-only on the server. You can also change the chmod option to customize the rights, like ensuring everyone has full control.

Install SQL Server 2022 with full-text search on Apple Silicon Macs

Michael Argentini Avatar
Michael ArgentiniThursday, April 27, 2023

It looks like a recent change to Docker has allowed Macs with Apple Silicon to run a full installation of Microsoft SQL Server 2022 with full-text search in a Docker container.

How is this possible? Docker has a new feature that can use Rosetta 2 for x64 emulation. That means it supports creating an x64-based Linux image/container and installing the free but full version of SQL Server with FTS.

Step 0: Enable Rosetta in Docker

Enable this feature in the Docker settings as seen in the image below. Look in Settings => Features in development.

Step 1: Create the dockerfile

Take the content below and save it as a text file named Dockerfile. This configuration creates an Ubuntu 20.04 image, then installs the Microsoft package repositories, SQL Server 2022, FTS, and MS Tools.

# Docker image with msssql 2022 with full text search enabled;
# Based on work in: https://github.com/Microsoft/mssql-docker

# Base OS layer: Latest Ubuntu LTS
FROM --platform=linux/amd64 ubuntu:focal

# Install prerequistes since it is needed to get repo config for SQL server
RUN export DEBIAN_FRONTEND=noninteractive && \
    apt-get update && \
    apt-get install -yq curl apt-transport-https gnupg && \
    # Get official Microsoft repository configuration
    curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add - && \
    curl https://packages.microsoft.com/config/ubuntu/20.04/packages-microsoft-prod.deb --output packages-microsoft-prod.deb && dpkg -i packages-microsoft-prod.deb && \
    curl https://packages.microsoft.com/config/ubuntu/20.04/mssql-server-2022.list | tee /etc/apt/sources.list.d/mssql-server.list && \
    apt-get update  && \
    # Install SQL Server from apt
    apt-get install -y mssql-server && \
    # Install optional packages
    apt-get install -y mssql-server-fts && \
    ACCEPT_EULA=Y apt-get install -y mssql-tools && \
    # Cleanup the Dockerfile
    apt-get clean && \
    rm -rf /var/lib/apt/lists

# Run SQL Server process
CMD /opt/mssql/bin/sqlservr

Step 2: Build the dockerfile

This will build the Dockerfile and create a Docker image named sqlserver.

docker build -t "sqlserver:latest" .

Step 3: Run the container

This will create a container from the new image and run it. The sa user will have a password of P@ssw0rdz!, the EULA will be accepted, and port 1433 will be opened.

docker run -d --platform linux/amd64 --name sqlserver -e 'ACCEPT_EULA=Y' -e 'SA_PASSWORD=P@ssw0rdz!' -p 1433:1433 sqlserver:latest

At this point you should have SQL Server running! You can also have it start up with Docker by executing the command below:

docker update --restart=always sqlserver

© 2026, Fynydd LLC / King of Prussia, Pennsylvania; United States / +1 855-439-6933

By using this website you accept our privacy policy. Choose the browser data you consent to allow:

Only Required
Accept and Close