Smarting up google docs and sheets

I’d like to talk about the growing experience with Google Docs and Google Sheets to use them for more complex needs and functions in this post.

It’s been a recent theme of the topic that I’ve been talking about ways to use collaborative cloud services for documentation purposes. Whether for personal or business/team communication. I’ve also talked about going beyond the need to have plain written form of documents to the smarter, more complex form of information forms in Smart(Er) Documents – Quip, Notion, Airtable, Coda Or Good Old GDocs & GSheets post.

I like Google docs from the availability perspective that it’s available without needing a complicated pricing structure and Google allowing google docs to be open for public google accounts that most people can access google docs with their personal Google accounts even if they don’t have company accounts (not using GSuite). I also like google docs is very familiar in the visual form that all of use are used to from Microsoft Office or other office software suites (Open, Libre). This also may look google docs products outdated compared to more modern collaborative editing tools like Notion, Quip.

All of these services have equally powerful APIs but probably not as robust as Google Docs. Google Docs also has a secret weapon that I’d like to introduce lightly in this post called Google Apps Script aside from their powerful API. This is a very wide and under-discussed topic online that gives Google Docs tools a huge edge. I may want to focus on this by talking sub-topics about the Google Apps Scripts down the road.

Google Apps Script

Google Apps script is a scripting/automation feature of google larger Google products including Gmail, google calendar, google drive and few other important google products you may be already using.

Google apps script runs on your browser (or mobile device) within the google tool you’re using. Scripts can register additional UI elements in the tools you use (register a new menu item), or watch/listen changes on the documents you create (events like, when a row is updated in google sheets) or you can even map your script parts to elements in the document content you are creating (such as buttons, dropdowns, checkboxes, etc…).

So far nothing new that other tools won’t do. Microsoft Excel and Word have its famous Macros available around ~20 years or more. Almost all other alternative software also have some form of automation that allows similar capabilities. The real power of Google Apps Script comes when you combine some of its online/cloud tools like google drive or google maps or Gmail with its documents. This makes the documents interactive with other services. Similar in the sense of using the other services APIs. One of the big things that makes me feel warmer to google docs tools are having real-time collaboration in your docs. This makes collaborative writing/editing experience superb. We often real-time write content with my team and I find the conversational aspect of the collaborative work priceless.

Google Apps Script is based on JavaScript. So you write Javascript snippets to define your custom functionality along with a large list of available google services APIs ready to use.

My favorite use cases of using Google Apps Script

Pull a dataset from our internal services or public sources dynamically to google sheets

We use this more often with our analytics services internally we use. These sheets are generally reports we create but often update with the latest version of the data.

Google sheets already have IMPORTDATA and IMPORTXML functions that pull a CSV or XML formatted data easily. But often we use a service that we haven’t built that doesn’t have CSV or XML exposure of its data. Often it’s a REST API returns JSON. You can use a helper function like https://github.com/bradjasper/ImportJSON or create your custom processor to pull the data and shape it to the way you want in google apps script. We often do the latter.

Add custom functions to google sheets

We use this a lot to create custom functions (generally pull data from other cloud tools we use), like getting Trello card details (title, status, assignee) i.e: =TRELLO(eio3u48d) or you can do other public services like getting weather forecast for a zip code =WEATHER(11222)

Send mails from google docs or sheets using your Gmail account

This goes into the automation of your workflow. As I mentioned above, with google apps script, you can map UI elements (menus or a button like an element in the document content) to a custom trigger in your script that does something for you. We sometimes create sheets or docs containing form-like formats or in the google sheet scenario, an action to be taken by the user for each row there is data. For this examples sake, think like a contact list with name, email and thank-you-note columns. We use google apps script to create a button-like action item in a column we define (let’s say it’s the next column to the thank you note) with label say “Send Thank You Note”. With google apps script, we can register this column to accept clicks and trigger a google apps script function. The google apps script function then can pull the clicked row number and the values in that row for the name, email and thank you note. Then with few lines of code, we can utilize gmail service api (without needing to do complicated SDK installations and -more importantly- deal with authentication) to send an email to the recipient with the content we want (in this case use the thank you note column as the email content. This is a huge convenience compared to building out this capability in a service or custom code from scratch.

Put a google sheet data to your calendar and update accordingly

Another great use case is to have timeline-based planning to be pushed to google calendar and update accordingly. We do this in a similar fashion as the previous scenario, but utilizing the google calendar service in the google apps script.

Wrapping up

There are many more interesting use cases for google apps script. There are also many communities created/maintained lists, directories for great google apps script examples and resources.

https://github.com/contributorpw/google-apps-script-awesome-list is a good start. Also checking GitHub tag search is worth looking https://github.com/search?q=topic%3Agoogle-apps-script

The official documentation of google apps script is here: https://developers.google.com/apps-script

Launch of my latest product Screenshot Tracker

🚨 I’ve just launched a new product (also in 😸 Product Hunt): 📸 Screenshot Tracker, a desktop all that helps capture full web page screenshots in different resolutions at once.

Screenshot Tracker helps you to collect:

  • 🖼 Full-size screenshots for your web pages
  • 🎳 Multiple URLs at once
  • 📱 In multiple resolutions (desktop, tablet portrait, landscape, and mobile)
  • 🔁 Repeat/Retake screenshots

with one click!

Screenshot Tracker is great for designers and developers to test their work while iterating on a web page design and development.

And it’s 100% free & open source. PRs welcome!

Project Homepage: https://screenshot-tracker.nomadinteractive.co/

Also, give me your support in Product Hunt page

Working with Memcache on your NodeJS app on Heroku

I wrote about Heroku in the past by showing my love for this amazing service. Heroku helps developers to start writing and deploying apps much easier than any other service. This allows developers to try things out and bootstrap their ideas in a matter of minutes. More about Heroku I talked in this article: https://mfyz.com/using-heroku-for-a-quick-development-environment/

Heroku Marketplace

I also mentioned before that Heroku has a 3rd party add-on marketplace. There are all types of application infrastructure, cloud services from fully hosted database services or CDNs to transactional email delivery services. Its marketplace is filled with countless gems that are worth a weekend to just explore these great services. That also works with Heroku within a few clicks/commands.

Why cache? Why Memcache?

One of them which is almost every distributed cloud application’s primary need is memory-based fast caching. There are different ways and models to store and use data from memory based caches but most common is for key-value based storage with TTL (time to live) which makes the data stored short-lived and cleaned up regularly.

The primary reason to use a memory-based cache is to optimize the performance of an application. In general terms, you cache frequently used data to cut time on queries to the database, reads to disks as long as you have a certain confidence that the data you are caching is not frequently changing within a short time while it lives in the cache. Even with the scenario that the data can be changed, there are simple ways to refresh the cache on the “update” events to make sure cached data is always valid and it cuts expensive operation time in the code flow.

As I mentioned, there are many different memory-based storage engines, protocols, services. One of the most simple yet effective and widely used engine is “Memcache”. It is an open-source tool that can be easily installed and hosted on any server that will run as a deamon and simply using an open port on a server to accept storage requests and queries. Memcache is designed to store and serve all of its data from the physical memory of the server. Therefore it’s very very fast and responsive when it’s combined with a high throughput network between the Memcache and application servers.

Heroku also supports multiple providers for Memcache and other memory-based storage engines. One particular service is very seamless to create an instance and set it up on a Heroku application, called Memcachier.

Memcachier

The memcachier instance free and can be added without any further account details from an external site. But it comes with very limited resources. It is only 25mb total cache size and 10 concurrent connection limit. Even though this number is small, for a node.js application, it will be mostly long-living. It will be more than enough if you are not planning to have many environments connecting to it at once. It is certainly enough to have a quick Memcache server to build stuff up and running. For more details about Memcachier service and its pricing model here is the Heroku elements page explaining all that: https://elements.heroku.com/addons/memcachier

To add this add-on to your application, in your project folder, run following command:

This will add memcachier to Heroku config as an environment variable. Copy the variable to your local .env file to be able to connect from your local development application.

To use memcache, on a node.js app, we will use a npm package called “memjs”. To install, run:

The usage of the package is pretty straightforward, here is the short example it’s quoted from memcachier’s Heroku get started guide:

Usage is as simple as using get and set async functions to set and get a value with a key.

I created a more visible version of this example with a relatively long, time-consuming mathematical calculation, so caching an expensive calculation (or data retrieval like a slow database query) to be cached in Memcache, so it’s calculated first, then when requested consecutively, it is served from Memcache instead of re-calculating. In my example, I used a calculation of a “primal” number which takes time.

https://github.com/mfyz/heroku-memcachier-example

Developing and Deploying Nodejs (Express) apps on Heroku

Heroku is an amazing platform for getting quick development up and running in a smart virtual instances. There is no hussle to get additional services you may need for a quick and dirty app to ground up. I’ve already wrote about how to use heroku for quick development environment before: https://mfyz.com/using-heroku-for-a-quick-development-environment/

This short article will be about specifically developing and deploying node.js and express apps on heroku. There is actually not much difference for deploying a node.js app than a php application or in another language. Heroku CLI tool automatically detects the application type from the package.json file for a node.js application and it’s entry point from there.

For the express related parts, just go ahead and see the very simple example I put up in github the past:
http://github.com/mfyz/heroku-nodejs-express-helloworld

Another more detailed express example that uses pug template engine for it’s layouts and views:
https://github.com/mfyz/express-pug-boilerplate

Aside of the application itself, there are few key points I found helpful when creating and deploying node.js apps. 

Use environment variables

Using environment variables is the best way to set configuration details for your application.

Setting which node.js version your app will use

As simple as adding “engines” object in package.json and having your node.js version defined in “node” property inside engines object like:

Same applies for npm and yarn versions to be defined within engines object as well.

Use prebuild and postbuild steps to prepare additional steps needed for your application build

By default, heroku will build your application on every deployment. This is not very meaningful for pure node.js applications but you app may need build. Like gulp, grunt, webpack builds. For this, heroku will read “build” npm script if exists in package.json. Aside of this, heroku will always install dependencies with npm install as a minimum build step. If you need additional steps before or after the build, you can define these in npm scripts as heroku-prebuild and heroku-postbuild named scripts.

Utilize heroku add-ons

Remember, Heroku comes with tons of 3rd party services which a lot of them have free packages that will be enough to try things out and start coding stuff up quickly. One of my favorite is heroku’s internal database service providing postgresql database with single command line command:

Wrapping up

All in all, heroku is a great cloud platform allow developers to kick off ideas, starting with simple code to grow into complex distributed applications very easily. In my opinion, it should be in the go-to tools for every engineer’s arsenal.

Importance of changelogs

It’s very important to keep a changelog and be disciplined about maintaining it as your product progresses. This applies to both product management and software development/management processes.

I want to mention a great source that talks, describe and almost accepted as a standard in open source community for changelogs:

https://keepachangelog.com

What is a changelog?

A changelog is a file that contains a curated, chronologically ordered list of notable changes for each version of a project.

Why keep a changelog?

To make it easier for users and contributors to see precisely what notable changes have been made between each release (or version) of the project.

Who needs a changelog?

People do. Whether consumers or developers, the end-users of software are human beings who care about what’s in the software. When the software changes, people want to know why and how.

An example Changelog (from Stretchly)

Github renders this markdown changelog beautifully: https://github.com/hovancik/stretchly/blob/master/CHANGELOG.md

Single JavaScript file node/express/Instagram authentication (OAuth) and get user photos

Get Ready

Register your Instagram app in their developer portal and obtain client id and client secret keys. To do that, follow the steps below:

  • Go to IG Developer page: https://www.instagram.com/developer/
  • Click “Register your application” button (if you are not logged in, you will be asked to log in on this step)
  • Fill all fields. The only field you need to pay attention to is the “valid redirect URLs”. This is where your app is publicly hosted. Below, we will create a URL on the application to capture Instagram authentication after the user goes to the Instagram page for permission dialog, then comes back to this URL. It’ll be something like https://yoursite.com/instagram/callback
  • Once you register the app, the page will display client id and secret. Keep this ready on the next steps.

Code it up

Let’s set up the plain node.js and express the application.

First, install the required packages:

index.js (or server.js)

Deploy and run

Either locally or after you place it on your server, run:

Tip 1: use “forever” on your server to run this application permanently.

Tip 2: For experimental purposes, you can run this app on your local and have a tunneling tool like “ngrok” to open your local port to the public with a quick domain assignment. Ngrok will provide a URL (random subdomain on their domain), you have to update the IG developer app’s settings to add this domain as a valid redirect URL, otherwise, after this app redirects user for authentication to Instagram, it will give an error.

Get the real thing

The code above in this article was a quick and dirty version. I put the little bit more proper express application version on Github. It uses pug for its views and has proper layout/content block separation as well.

https://github.com/mfyz/express-passport-instagram-example

Make your car smarter with Automatic

After I got my last car about 8 months ago, I’ve invested some money to make my car smarter like getting Apple car play installed. One of the things I did was to install a mini device to my car that makes my car a little bit smarter. The device name is Automatic.

This tiny gadget connects to the car’s diagnostic port which it receives it’s power and read and monitor car’s data like engine lights and stuff. The device contains GPS tracker and a sim card to continuously have low-speed connectivity like GPRS. This is more than enough to store it’s monitoring data in the cloud.

Here is the cool stuff you can do with having automatic on your car:

  • Parking tracking – probable the best feature. I hate the idea of trying to remember where I parked in the street. This removes the need for that. Open the app and check where the car is 🙂
  • Track when you started and finish driving. It’s like anti-theft alarm. You get a notification if someone starts the engine – sweet
  • Track the whole tour you drove. It draws the driving path on the map and you can see your driving history like uber receipts.
  • These driving records also contain avg speed, gas consumption…
  • Driving style – how often you stop. how aggressive you accelerate…
  • IFTTT integration – this opens the possibilities 100 times. You can set triggers like driving away from am geofence, or arriving at the geofence. This allows setting smart behaviors like Turn on garage lights when I arrive home.

It’s a little bit of a novelty features but if you are data nerd like me, it’s automatic tracking of your driving data which is enough value to me.

https://automatic.com

Time to take a break from your computer

I firmly believe that having regular breaks from long work sessions is a necessity. We work sitting long hours without much movement. It’s often we forget to take long or short breaks when we work on a task that occupies our brain. Also sometimes it’s hard to break a focus session.

Without a tool reminding us of the breaks, it’s very difficult to track when we started working and when to take a break. GTD methods like Pomodoro helps to track these moments but even with planned natural work blocks we miss taking a break from the screen.

I started to use a tool to remind break time with a schedule. First I used a paid app called Timeout.app. Then one of my colleagues found the open source alternative of the same tool. https://github.com/hovancik/stretchly

I’ve forked and modified the look and feel and some of the element’s positions for my own taste https://github.com/mfyz/stretchly

I highly suggest a timeout app like this one to be embedded in the workflow. Now I take much more regular breaks with the help of a tool like this. Sometimes I feel it becomes annoying because I feel I’m very focused. I don’t want to break the focus session but I also acknowledge the importance and the value of giving opportunity and space to our minds with these breaks.

The app is pretty straight forward to use. You set micro and regular breaks with an interval. I take 10 seconds micro break in every 10 minutes. Every after 2 micro breaks, I take a regular break of 5 minutes (every 30 minutes)

https://github.com/mfyz/stretchly

Mobile simulators on cloud

I had a radical switch to iPad Pro as my primary workspace and I used that set up exclusively for 6 months. As part of my day to day work, I test a lot of features and new developments on our mobile development cycle. I uninstall and re-install and replicate a lot of weird conditions on mobile environments. It’s great to do these and relatively easy to simulate these cases with real devices but when it comes to testing Android, it gets a pretty wide range of devices and hardware/software differences. Often I test the same user behavior on 5-6 devices which is very annoying.

This could be easily me doing testing 🙂

Anyway, I was in search of doing this on the actual mobile device but replicate the other hardware and software variations but in the very cloud, remote-desktop fashion. Why there is no strong service doing this more professionally? Other than testing purposes, I really want to open and want to play with a new android OS on a real device with any resolution I want to and stream that to my existing mobile device? I would love to check out some android apps on my ios device, vice versa…

I gave a try to:

  • AWS Device Farm – their browser experience is very lagging, starting up a device takes at least 4-5 minutes (why?) and there are most of the time device initialization errors.
  • appetize.io – so far, the best experience, mobile browser friendly as well, but limited virtual device variations and there is just one real device type.
  • Genymotion cloud – I haven’t got a chance to play with it since they are in their private beta.
  • Other option is to build it manually with VPN or RDP but required rooting or jailbreaking the device which is not ideal and painful to go through.

Hopefully, we’ll get there and we’ll have the option to stream apps or OSes as we do movies tv shows today…