Category: Development

Entity Framework Decimal Scale and Precision Convention

Entity Framework Decimal Scale and Precision Convention

Entity Framework very much relies on conventions in order to accomplish a lot of what it does. In many cases it can be a hands-off tool: so long as you follow its conventions you should be in the clear. What do you do when the convention just isn’t cutting it?

Entity Framework Rounding to Two Decimal Places

I recently created a table that had a column setup as DECIMAL(19, 4).  That’s a precision of 19 and a scale of 4. Using Entity Framework I created a record in the table where I attempted to utilize the allotted scale (for example, 20.0015). The save was successful, but to my surprise the persisted number was rounded to two decimal places. It turns out this is by design, as Entity framework defaults to saving two decimal places. 

Entity Framework Decimal Scale and Precision Convention

Fortunately there’s a straightforward approach to circumventing this behavior by defining a new convention for Entity Framework to follow when dealing with any given property. Let’s see what that looks like.

Create the Decimal Precision Attribute Class

A new attribute will be created that serves the purpose of defining a precision and scale. This attribute can later be used to decorate a property of an entity object, which we’ll configure EF to recognize.

Add a new class DecimalPrecisionAttribute with the following:

[AttributeUsage(AttributeTargets.Property, Inherited = false, AllowMultiple = false)]
public sealed class DecimalPrecisionAttribute : Attribute
{
    public DecimalPrecisionAttribute(byte precision, byte scale)
    {
        Precision = precision;
        Scale = scale;
    }

    public byte Precision { get; set; }
    public byte Scale { get; set; }
}

Create the Decimal Precision Attribute Convention Class

The attribute is created, it’s time to make a new decimal convention class which will later be wired into the overall conventions Entity Framework is aware of.

Add a new class DecimalPrecisionAttributeConvention with the following:

public class DecimalPrecisionAttributeConvention
    : PrimitivePropertyAttributeConfigurationConvention<DecimalPrecisionAttribute>
{
    public override void Apply(ConventionPrimitivePropertyConfiguration configuration, DecimalPrecisionAttribute attribute)
    {
        if (attribute.Precision < 1 || attribute.Precision > 38)
        {
            throw new InvalidOperationException("Precision must be between 1 and 38.");
        }

        if (attribute.Scale > attribute.Precision)
        {
            throw new InvalidOperationException("Scale must be between 0 and the Precision value.");
        }

        configuration.HasPrecision(attribute.Precision, attribute.Scale);
    }
}

Add Convention to the Model Builder

Now that we’ve created a new convention to handle our precision and scale needs, we need to add it to the model builder. 

In your DbContext class, override the OnModelCreating method with the following:

protected override void OnModelCreating(DbModelBuilder modelBuilder)
{
    modelBuilder.Conventions.Add(new DecimalPrecisionAttributeConvention());
}

Decorate a Property with the Decimal Precision Attribute

We’ve created a property to define precision and scale and added a convention to Entity Framework to use this precision and scale when found. Now all we have to do is decorate a property on one of our entity objects. Here’s what that might look like:

public class PayInfo {
    [DecimalPrecision(19, 4)]
    public decimal Rate { get; set; }
}

There you have it: properties decorated with the DecimalPrecision attribute will now persist with the predefined precision and scale.

Setting Opacity of Android Toolbar Overlay Based on ViewPager Position

Setting Opacity of Android Toolbar Overlay Based on ViewPager Position

I’ve recently delved into the world of mobile development using Xamarin with Visual Studio. A tutorial on their website introduced me to the Android toolbar, and how it can be used as a more robust and flexible action bar. This was fairly straightforward to implement but I had a requirement that the toolbar overlay shouldn’t be visible on certain fragments of a ViewPager.

The Goal

What were trying to achieve here is a way to change the opacity of the toolbar overlay based on the ViewPager’s current position and offset. As paging occurs, the toolbar should become more or less transparent depending on the fragment we’re paging to. Here’s an example of what I mean:

Android Toolbar Overlay with ViewPager

The above is what we’ll be covering in this post. Here’s some additional resources that helped get me to this point. These go a bit more in depth in their respective areas and will prove useful in case anything here isn’t quite detailed enough:

The full solution has been added to my GitHub repository as well. Get to forking!

Let’s get to it!

Prerequisites

Before getting started you’ll need to create a new Xamarin Android project in Visual Studio. You’ll also need to add the v13 and v4 Android Support libraries as references.

While this post is targeted toward Xamarin for Visual Studio, the majority of its content is very translatable to Xamarin and Android Studio

In Visual Studio, adding the support libraries can be done via the NuGet Package Manager:

  1. Navigate to Tools -> NuGet Package Manager -> Manage NuGet Packages for this Solution...
  2. Click the Browse window, then search for Xamarin.Android.Support.v13
  3. In the resulting window, install both Xamarin.Android.Support.v13 and Xamarin.Android.Support.v4 libraries

Android Support Libraries NuGet Package Manager

Adding the ViewPager and Fragment Layouts

We’ll start by creating the ViewPager and its accompanying fragments which will enable that smooth swiping transition between the fragment views. This requires that we create a couple new fragments to be used by the ViewPager control, and add the ViewPager itself to the Main.axml layout.

Under the Resources -> layout folder of your project let’s modify Main.axml and add a couple new layouts:

Modify the Main Layout

This is the starting view of our activity and will encompass our toolbar overlay and the ViewPager. For now we’re just going to focus on adding the ViewPager. We’ll circle back to adding the toolbar overlay to this layout once we’ve built it.

Double-click the Main.axml layout file to edit it. The layout will likely open in the visual designer so you’ll want to click the Source button at the bottom left of the designer, then paste the following:

<?xml version="1.0" encoding="utf-8"?>

<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
  android:layout_width="match_parent"
  android:layout_height="match_parent">
    <android.support.v4.view.ViewPager
      android:id="@+id/viewPager"
      android:layout_width="fill_parent"
      android:layout_height="fill_parent" />
</RelativeLayout>

Add the FragmentOne Layout

This layout will be the one where we don’t want the toolbar overlay to be visible, allowing the view itself to utilize the screen in its entirety. An example of where this might be handy is when showing a live camera stream in the view; you wouldn’t want the toolbar overlay to take precious real-estate.

Add a new layout file called FragmentOne under the Resources -> layout folder with the following inside:

<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
  android:layout_width="fill_parent"
  android:layout_height="fill_parent"
  android:background="#CCC">
    <TextView
      android:layout_width="wrap_content"
      android:layout_height="wrap_content"
      android:text="This view should HIDE the toolbar."
      android:textColor="#000"
      android:textSize="14dp"
      android:layout_centerHorizontal="true" />
</RelativeLayout>

Add the FragmentTwo Layout

This layout will embrace the toolbar overlay. Since the toolbar will indeed be an overlay (that is, it’s floating on top of other content, rather than pushing other content below), we’ll need to force the view’s content to begin below the toolbar. This can be achieved in a number of ways, but for this example we’re just adding padding to the top of the fragment.

Add a new layout file called FragmentTwo under the Resources -> layout folder with the following inside:

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
  android:layout_width="match_parent"
  android:layout_height="match_parent"
  android:background="#FFF"
  android:paddingTop="?android:attr/actionBarSize">
  <TextView 
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:text="This view should SHOW the toolbar."
    android:textColor="#000"
    android:textSize="14dp"
    android:padding="5dp"
    android:layout_centerHorizontal="true" />
</LinearLayout>

Implement the FragmentPagerAdapter for the ViewPager

At this point we have the fragment layouts for our ViewPager in place and the ViewPager itself added to the main layout. Now we need to implement a FragmentPagerAdapter for our ViewPager so it knows about our fragments, and how to view them.

Create the ViewPageFragmentAdapter Class

As I mentioned, the ViewPager control requires a concrete implementation of the FragmentPagerAdapter so it knows how to handle paging. In your project, add a new class called ViewPageFragmentAdapter with the following:

 public class ViewPageFragmentAdapter : FragmentPagerAdapter {
    public ViewPageFragmentAdapter(FragmentManager fragmentManager, List<Fragment> fragments) : base(fragmentManager) {
        this.fragments = fragments;
    }

    private readonly List<Fragment> fragments;

    public override int Count => fragments.Count;

    public override Fragment GetItem(int position) {
        if (position < 0) {
            position = 0;
        }

        if (position >= Count) {
            position = Count - 1;
        }

        return fragments[position];
    }
 }

To resolve the missing reference error you’re seeing on the FragmentPagerAdapter, add the using statement using Android.Support.Android.V13.App;.

Modify the MainActivity Class

Our MainActivity class needs to be updated to do the following:

  • Grab a reference to our ViewPager
  • Implement the ViewPager.IOnPageChangeListener interface, which is responsible for handling ViewPager paging and will eventually be where we set the opacity of our toolbar overlay
  • Set the ViewPageFragmentAdapter as the official adapter of our ViewPager.

Modify your MainActivity class to look as follows:

[Activity(Label = "ToolbarOverlay", MainLauncher = true)]
public class MainActivity : Activity, ViewPager.IOnPageChangeListener {
    public void OnPageScrolled(int position, float positionOffset, int positionOffsetPixels) {
    }

    public void OnPageScrollStateChanged(int state) {
    }

    public void OnPageSelected(int position) {
    }

    private ViewPager viewPager;
    protected override void OnCreate(Bundle savedInstanceState) {
        base.OnCreate(savedInstanceState);

        // Set our view from the "main" layout resource
        SetContentView(Resource.Layout.Main);

        var fragments = new List<Fragment> {
            new FragmentOne(),
            new FragmentTwo(),
        };

        viewPager = FindViewById<ViewPager>(Resource.Id.viewPager);
        viewPager.Adapter = new ViewPageFragmentAdapter(FragmentManager, fragments);
        viewPager.AddOnPageChangeListener(this);
    }

}

To resolve the reference error you’re seeing on ViewPager.IOnPageChangeListener, add the using statement using Android.Support.V4.View.

At this point we’ve created a ViewPager, a couple fragments for it to page through and implemented the necessary code to teach the pager how to page. With all that in place we should have a fully functional pager, but the default action bar is still visible. Let’s replace it with our custom toolbar overlay!

Replacing the Default Action Bar with a Toolbar Overlay

I mentioned it before, but this guide on Xamarin’s website helped walk me through how to replace the action bar with a custom toolbar. We’ll more or less be following its implementation with a few adjustments.

Create a Custom Theme to Disable the Action Bar

Before we can use our custom toolbar overlay, we need to disable the default action bar. Disabling the action bar can be done by implementing a custom theme. In your project under the Resources -> values folder, add a new XML file called styles.xml with the following:

<?xml version="1.0" encoding="utf-8" ?>
<resources>
  <style name="StoriKnowTheme" parent="@android:style/Theme.Material.Light.DarkActionBar">
    <item name="android:windowNoTitle">true</item>
    <item name="android:windowActionBar">false</item>
    <item name="android:colorPrimary">#5A8622</item>
  </style>
</resources>

That wasn’t too bad, but we still need to set this as the theme in the the Android App Manifest file before our application starts using it. In your project, open the AndroidManifest.xml file under the Properties tree and add the android:theme="@style/StoriKnowTheme" attribute to the application node. Here’s what mine looks like:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android" 
          package="ToolbarOverlay.ToolbarOverlay" 
          android:versionCode="1" 
          android:versionName="1.0">
  <uses-sdk android:minSdkVersion="21" />
  <application android:theme="@style/StoriKnowTheme" android:allowBackup="true" android:label="@string/app_name">
  </application>
</manifest>

Create the Page Aware Custom Toolbar Overlay

We’re almost there! At this point we just need to:

  • Create the custom toolbar overlay layout
  • Add this layout to the Main.axml file
  • Set the opacity as we swipe between pages

Under the Resources -> layout folder, add a new layout called Toolbar with the following:

<?xml version="1.0" encoding="utf-8"?>
<Toolbar xmlns:android="http://schemas.android.com/apk/res/android"
    android:id="@+id/toolbar"
    android:layout_width="match_parent"
    android:layout_height="wrap_content"
    android:minHeight="?android:attr/actionBarSize"/>

Now open up the Main layout under Resources -> layout and include the toolbar overlay you just created (that’s line 11 below):

<?xml version="1.0" encoding="utf-8"?>

<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
  android:layout_width="match_parent"
  android:layout_height="match_parent">
    <android.support.v4.view.ViewPager
      android:id="@+id/viewPager"
      android:layout_width="fill_parent"
      android:layout_height="fill_parent" />
  <include android:id="@+id/toolbar" layout="@layout/Toolbar" />
</RelativeLayout>

Great, Main.axml knows about the toolbar now, but we still need to tell the main activity that this is actually the new action bar. Along with that, we want to set the opacity of the toolbar overlay as we change pages.

Use the Toolbar as an Action Bar and Set its Opacity During Paging

To wrap things up, let’s modify the MainActivity one last time using the highlighted rows as a visual guide to what changed:

[Activity(Label = "ToolbarOverlay", MainLauncher = true)]
public class MainActivity : Activity, ViewPager.IOnPageChangeListener {
    private ViewPager viewPager;
    private Toolbar toolbar;

    public void OnPageScrolled(int position, float positionOffset, int positionOffsetPixels) {
        var opacity = 1.0;
        int toolbarTranslationZ = 100;
        int viewPagerTranslationZ = 99;
        if (position == 0) {
            if (positionOffset == 0) {
                //  position and positionOffset of 0 means we're on the first fragment, which should
                //  have the viewpager on top of the toolbar, so we set the z-index of the toolbar 
                //  to be slightly less than the view pager.
                toolbarTranslationZ = 99;
                viewPagerTranslationZ = 100;
            }

            opacity = positionOffset;
        }

        //  the first parameter of the Color.Argb method is what sets the opacity level
        toolbar.SetBackgroundColor(Color.Argb((int)(opacity * 255), 200, 84, 59));
        toolbar.SetTitleTextColor(Color.Argb((int)(opacity * 255), 255, 255, 255));

        //  just setting the opacity will hide the toolbar, but we'll also set its
        //  z-index for good measure, and to guarantee it doesn't intercept touch events
        //  while it's invisible.
        ViewCompat.SetTranslationZ(viewPager, viewPagerTranslationZ);
        ViewCompat.SetTranslationZ(toolbar, toolbarTranslationZ);
    }

    public void OnPageScrollStateChanged(int state) {
    }

    public void OnPageSelected(int position) {
    }

    protected override void OnCreate(Bundle savedInstanceState) {
        base.OnCreate(savedInstanceState);

        // Set our view from the "main" layout resource
        SetContentView(Resource.Layout.Main);

        var fragments = new List<Fragment> {
            new FragmentOne(),
            new FragmentTwo(),
        };

        toolbar = FindViewById<Toolbar>(Resource.Id.toolbar);
        SetActionBar(toolbar);
        ActionBar.Title = "StoriKnow Toolbar";

        viewPager = FindViewById<ViewPager>(Resource.Id.viewPager);
        viewPager.Adapter = new ViewPageFragmentAdapter(FragmentManager, fragments);
        viewPager.AddOnPageChangeListener(this);
    }

    public class ViewPageFragmentAdapter : FragmentPagerAdapter {
        public ViewPageFragmentAdapter(FragmentManager fragmentManager, List<Fragment> fragments) : base(fragmentManager) {
            this.fragments = fragments;
        }

        private readonly List<Fragment> fragments;

        public override int Count => fragments.Count;

        public override Fragment GetItem(int position) {
            if (position < 0) {
                position = 0;
            }

            if (position >= Count) {
                position = Count - 1;
            }

            return fragments[position];
        }
    }
}

Conclusion

There you have it, we’ve successfully utilized a custom toolbar overlay on a view pager and tied its opacity level to the current pages position and offset.

Working Around the Android Device Monitor File Explorer Bug

Working Around the Android Device Monitor File Explorer Bug

Using Xamarin and Visual Studio I created a sample application to test out the new Camera2ApI. The application is straightforward: provide a stream from the camera and persist the image when a button is clicked. After the image is saved it’d be nice to actually see it on the file system; that’s where the Android Device Monitor comes in. The Android Device Monitor is a debugging and analysis tool which provides a file explorer that enables you to navigate your device’s system. Unfortunately, my file explorer always returned an empty result.

After researching a bit I found I wasn’t the only one experiencing this issue. This has even been reported to the Android team and accepted as a known issue. There hasn’t been a bug fix released to date, but there is a workaround which I’ll be covering in this post.

Create a Supported Android Virtual Device

This bug was introduced in the API 24 release and still exists today. Fortunately, we can still make use of the file explorer by creating a new Android Virtual Device that targets API 23 or earlier. These instructions will be geared toward a Visual Studio audience, but should translate well to the other platforms:

Already have a Virtual Device that targets API 23 or less? Fantastic, you can skip these steps entirely and use that virtual device when you need the file explorer.

  1. In Visual Studio, open the Android SDK Manager: Tools -> Android -> Android SDK ManagerOpen Android SDK Manager
  2. Under any API that’s less than 24 (I chose API 21 here), install the SDK Platform and at least one System ImageAndroid SDK Manager Install
  3. Open the Android Emulator Manager (also called the Android Virtual Device Manager): Tools -> Android -> Android Emulator ManagerOpen Android Emulator Manager
  4. On the right-hand side of the Android Emulator Manager, click the Create button to begin creating a new virtual device. You may create an entirely different device than pictured below, and that’s alright! The point here is to create one that targets API 23 or less.Create Android Virtual Device

Now that we have a virtual device created, let’s verify that our solution works.

Verify the Android Device Monitor File Explorer Works

If all goes well we should be able to explore the files on our new virtual device using the Android Device Monitor file explorer.

  1. In the Android Virtual Device Manager (Tools -> Android -> Android Emulator Manager), start your newly created virtual deviceStart a Virtual Device in Android Device Manager
  2. Once the emulator starts, navigate to the Android Device Monitor: Tools -> Android -> Android Device Monitor, select the loaded emulator (emulator-5554 below) and view the File Explorer tab
    Android Device Monitor File Explorer

There we have it, the file explorer is back in action!

Conclusion

The Android Device Monitor is a highly valuable weapon in our arsenal, and will continue to be so. Bugs like this happen and will eventually go away, but until then we have a viable workaround for those of us who are able to target a lower API.

Map to a Primitive Type from a Complex Type Using AutoMapper

Map to a Primitive Type from a Complex Type Using AutoMapper

Using AutoMapper, I created a mapping for an object with many complex properties, one of which I wanted to map to a primitive type. Here’s how I configured the map from the child property to the primitive type:

CreateMap<ComplexType, int>()
    .ForMember(dest => dest, opt => opt.MapFrom(src => src.Id));

This looks like your standard map configuration: take the Id from the complex property and map it to an int. However, when attempting to run the application I received an AutoMapperConfigurationException:

Custom configuration for members is only supported for top-level individual members on a type.

The stack trace pointed to the CreateMap<ComplexType, int> configuration above. This isn’t a one-to-one mapping, I’m trying to reduce a complex object into a primitive, which AutoMapper requires some help from us to understand. For this, we’ll need to use a custom type converter.

 

Using Custom Type Converter to Map to a Primitive Type

The custom type converter allows you to take full responsibility for defining how one type maps to another. An example of this is mapping a string that looks like an int to an int.

You can create a full-fledged custom type converter by implementing the ITypeConverter<TSource, TDestination> interface, but I used the Func<TSource, TDestination> overload like so:

CreateMap<ComplexType, int>()
    .ConvertUsing(src => src.Id);

With the ConvertUsing extension in place, the mapper configuration no longer throws an exception.

Get Instagram Posts by Tag Name

Get Instagram Posts by Tag Name

Instagram has a very robust API which will enable you to integrate with it in just about any way imaginable. However, there’s also some public endpoints that may get you all the information you need. One such public endpoint which we’ll be covering here is the ability to grab Instagram posts by tag name without the need to register for the Instagram API.

Goal

It’s good to have goals. Ours are simple:

  • Retrieve Instagram posts by tag name as JSON using the public endpoint. 

Get Instagram Posts by Tag Name as JSON

If you were to navigate to the following URL, you’d be shown public Instagram posts with the tag name selfiehttps://www.instagram.com/explore/tags/selfie/

This is a nice way to browse all posts by tag, starting with the most recent. Now, try adding ?__a=1 to the end of that same URL: https://www.instagram.com/explore/tags/selfie/?__a=1

Pretty interesting, right? By adding ?__a=1, you’re telling Instagram to return the results as JSON, which provides a universal way for us to digest the information with code. 

Why does Instagram allow this? Great question, if you have the answer I implore you to share in the comments.

Making Sense of the JSON Result

Now that we know how to grab a JSON result of the most recent Instagram posts by tag name, we just need to make sense of it. We’ll start by using an online JSON pretty printer to make the JSON blob a bit friendlier to look at. There’s a lot of data, but we’re going to hone in on the nodes property under tag > media. This nodes property contains a breakdown of actual post data.

Below, I’ve highlighted some of the more interesting properties of the post that we’ll be discussing:

{  
   "tag":{  
      "name":"selfie",
      "content_advisory":null,
      "media":{  
         "nodes":[  
            {  
               "comments_disabled":false,
               "id":"1638021685389222559",
               "dimensions":{  },
               "owner":{  
                  "id":"4736824623"
               },
               "thumbnail_src":"https://scontent-iad3-1.cdninstagram.com/t51.2885-15/e35/c0.10.500.500/22860707_489313221440833_4558795553466482688_n.jpg",
               "thumbnail_resources":[  ],
               "is_video":false,
               "code":"Ba7bAtUAuKf",
               "date":1509487423,
               "display_src":"https://scontent-iad3-1.cdninstagram.com/t51.2885-15/e35/22860707_489313221440833_4558795553466482688_n.jpg",
               "caption":"#girl #style #pretty #blogger #beautiful #beauty #fashion #ootd #makeup #instagood #like4like #cool #moda #trend #adidas #shoes #shopping #goals #selfie #inspiration #outfit #look #fit #luxury #nails #model #followme",
               "comments":{  
                  "count":0
               },
               "likes":{  
                  "count":1
               }
            },
            ...

Referencing the highlighted properties:

  • code can be used to get information about the originator of the post. You may have noticed there’s an owner property, too, but for the sake of this article we’ll focus on the code property to get originator information.
  • date is when the post was submitted, and is represented by the number of milliseconds since the epoch.
  • display_src is the URL to the posted image. You can see the image by pasting that URL in your browser.
  • caption is, well, the caption associated with the post. This particular caption is a massive list of tags, one of which happens to be #selfie!

As I mentioned, the code property can be used to obtain information about the originator of the post. This is accomplished by plugging the value into another public API. Replace the <code> tag below with your code property value, and paste the URL in your browser to obtain another block of JSON:

https://www.instagram.com/p/<code>/?__a=1

We are once again provided with a decent chunk of data. Use the online JSON pretty printer we discussed earlier to format it so it’s easier to follow. Here’s a sample of the output, which I’ve shortened a bit:

{  
   "graphql":{  
      "shortcode_media":{  
         "__typename":"GraphImage",
         "id":"1638021685389222559",
         "shortcode":"Ba7bAtUAuKf",
         ...
         ...
         ...
         "owner":{  
            "id":"4736824623",
            "profile_pic_url":"https://scontent-iad3-1.cdninstagram.com/t51.2885-19/s150x150/18161617_288442401613475_2938897407410176000_a.jpg",
            "username":"andysprite",
            "blocked_by_viewer":false,
            "followed_by_viewer":false,
            "full_name":"Andy",
            "has_blocked_viewer":false,
            "is_private":false,
            "is_unpublished":false,
            "is_verified":false,
            "requested_by_viewer":false
         },
         ...

The post owner’s information is under the graphql > shortcode_media > owner property. From there you can get information such as username and full_name, which I’ve highlighted above.

We now have a fairly straightforward way to:

  • Grab the most recent Instagram posts by tag name as JSON.
  • Identify individual post data, and use that data to obtain additional information such as the poster’s name and username.

This is pretty great, we have a lot to work with here. However, the JSON results are paged which means at this point we’ve only looked at the very first set of results. A bit more investigation into the JSON result shows that paging is built in.

Requesting the Next Page of Results

Looking back at the JSON under the tags > media property, you’ll see another property called page_info (it may be easier if you collapse the nodes property, as that can take a lot of vertical space):

{  
   "tag":{  
      "name":"selfie",
      "content_advisory":null,
      "media":{  
         "nodes":[  ],
         "count":321717673,
         "page_info":{  
            "has_next_page":true,
            "end_cursor":"J0HWfjzPwAAAF0HWfjzOAAAAFiYA"
         }
      },
      "top_posts":{  }
   }
}

The page_info property has its own children, has_next_page and end_cursor. You probably guessed this, but if has_next_page is set to false, we’re at the very end of the results. Otherwise, the end_cursor property can be appended to the original URL we used via the max_id query parameter to retrieve the very next set of results. Here’s an example of what that might look like:

https://www.instagram.com/explore/tags/selfie/?__a=1&max_id=J0HWfjzPwAAAF0HWfjzOAAAAFiYA

That’s all there is to paging the result set.

Conclusion

We discussed using a public API to retrieve Instagram posts by tag name as a JSON result. After some investigation, we were able to understand the structure of this result and identify some of the more exciting properties. We’re now equipped with enough information to consume the JSON result in code. 

Linux Cronjob Doesn’t Execute Valid Python Script

Linux Cronjob Doesn’t Execute Valid Python Script

I’ve been using cron to execute a Python script used by my Raspberry Pi automatic cat feeder for some time now. After coming up with another script that needs to run on a regular schedule I added a second cronjob entry into the crontab.

That’s easy enough: first, in a terminal open the crontab to reveal existing jobs:

sudo nano crontab -e

Add the cronjob to the crontab by creating a new entry:

* * * * * /home/pi/Documents/Development/CatFeeder.py
* * * * * /home/pi/Documents/Development/Maintenance/Monitoring.py

The CatFeeder.py script has been running without a hitch, while the Monitoring.py script is the one being added.

After saving the file (this is nano, so CTRL+X, Y, Enter to save and exit) I expected the job to run every 60 seconds. However, nothing seemed to be happening. The new script never successfully executed. Meanwhile, the CatFeeder.py script was running flawlessly.

The script is an executable, I made sure of that by running chmod +x Monitoring.py, so that’s not the issue. I double-checked that the script ran without errors by invoking it manually: sudo python Monitoring.py. To no surprise the script ran as expected.

If the script runs fine, and a cronjob I already have in the crontab runs fine, then what gives?

SheBang #!: Specifying the Language Interpreter

There’s a very interesting line of code, or a comment rather, that I’ve noticed floating around in most of the Python scripts I’ve seen:

#!/usr/bin/env python

My tiny brain paid this line little attention at first, “how important could it be”, I thought. Enter the shebang #! character sequence. This sequence is treated as an interpreter directive by program loaders such as cron. That is, it specifies which interpreter to use when executing the script. 

Without this line of code the cronjob will not automatically use Python as the interpreter and will fail to execute the script. Add this line to the beginning of your script and watch your cronjob flourish!

What Was That You Said About the Script Running Without Issue?

It was previously mentioned that the Monitoring.py script executed without issue when invoked manually, like so:

sudo python Monitoring.py

This does indeed run without issue, even without the shebang #! line added to the file. However, the script name is preceded with the interpreter to use, python. If we execute the script a different way we’ll more accurately see what’s happening when the cronjob runs it.

In the terminal use the ./ command to execute the script:

./Monitoring.py

Without the shebang #!, Python is not used as the interpreter, and errors are shown in the terminal:

pi@raspberrypi:~/Documents/Development/Maintenance $ ./Monitoring.py
./Monitoring.py: line 2: import: command not found
./Monitoring.py: line 3: syntax error near unexpected token `1,'
./Monitoring.py: line 3: `sys.path.insert(1, os.path.join(sys.path[0], '..'))'

Now add the shebang #!/usr/bin/env python at the beginning of the script and execute it again. Unless your script has other syntactical problems it should run successfully.

Controlling a Servo Motor with Gmail and Raspberry Pi

Controlling a Servo Motor with Gmail and Raspberry Pi

You covered part one and part two (good work!), or maybe you didn’t and you found yourself here anyway (welcome). Part three is focused on the servo motor, primarily making it spin. Later in part four we’ll wrap things up by connecting it to the dispenser.

As always, let’s start with what it is we’re trying to accomplish here:

  • Connect the servo motor to the Raspberry Pi’s GPIO header pins
  • Write some code to make that motor functional
  • Tie in the GmailWrapper.py script we wrote in part two so the motor spins when the right email lands in your inbox

Here we go!

Connecting the Servo Motor to the Raspberry Pi

If you’re using the same servo motor as me, it has power, ground and control wires. If you decided to use the standard non-continuous servo motor (which is a viable alternative since we won’t be performing a full revolution), then: power is red, ground is black and white is control.

Connect your male-to-female jumper cables to the servo. Now, using the GPIO diagram below as reference, connect the wires to the Pi’s GPIO headers as follows:

Note: this diagram is for Raspberry Pi’s with 40 pin GPIO, but the below works with 26 pin GPIO as well.

  • power connects to header pin 2 (5v power)
  • ground (black wire for standard servo) connects to header pin 6 (Ground)
  • control (white for standard servo) connects to header pin 12 (GPIO18)

GPIO Pinout Diagram

Here’s what mine looks like (don’t mind the zip tie):

Servo Connecting to Pi
Servo Connecting to Pi’s GPIO

 

Writing Code to Make the Servo Motor Spin

Before getting started, let’s update our Pi to the latest packages and install the GPIO library. Run this in the terminal:

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install rpi.gpio

We’re all set. In the terminal, create a new python script using nano: sudo nano CatFeeder.py

Within the CatFeeder.py script, add the following code:

#!/usr/bin/env python

import RPi.GPIO as GPIO
import time

def feed():
    # let the GPIO library know where we've connected our servo to the Pi
    GPIO.setmode(GPIO.BCM)
    GPIO.setup(18, GPIO.OUT)

    try:
        servo = GPIO.PWM(18, 50)
        servo.start(12.5)

        # spin left, right, then left again rather than in a continuous circle
        # to prevent the food from jamming the servo
        for index in range(0, 3):
            dutyCycle = 2.5 if (index % 2 == 0) else 12.5
            servo.ChangeDutyCycle(dutyCycle)
            # adjust the sleep time to have the servo spin longer or shorter in that direction
            time.sleep(0.8) 
    finally:
        # always cleanup after ourselves
        servo.stop()
        GPIO.cleanup()

if __name__ == '__main__':
    # kick off the feeding process (move the servo)
    feed()

Save and exit the nano editor: CTRL+X, then Y, then Enter. Moment of truth, time to make that servo move! Let’s drop back into the Python interpreter, in the terminal:

# remember, hit enter after each line to have the interpreter... interpret
python
import CatFeeder
CatFeeder.feed()

I hope that went smooth for you, because I just got really excited. Welcome to physical computing! You moved a physical object with code, hell yeah.

GmailWrapper, Meet CatFeeder: Putting it all Together

Alright, so we’ve created a way to read emails, and we’ve created a way to move a servo motor. Now we need to combine the two so the servo motor moves when we read emails.

Open CatFeeder.py in nano and add the highlighted lines of code: sudo nano CatFeeder.py

#!/usr/bin/env python

from GmailWrapper import GmailWrapper

import RPi.GPIO as GPIO
import time

HOSTNAME = 'imap.gmail.com'
USERNAME = '<your gmail username>'
PASSWORD = '<your app password or regular gmail password>'

def feedByGmail():
    gmailWrapper = GmailWrapper(HOSTNAME, USERNAME, PASSWORD)
    ids = gmailWrapper.getIdsBySubject('feed cats')
    if(len(ids) > 0):
        try:
            feed()
            gmailWrapper.markAsRead(ids)
        except:
            print("Failed to feed cats, they're starvingggg")

def feed():
    # let the library know where we've connected our servo to the Pi
    GPIO.setmode(GPIO.BCM)
    GPIO.setup(18, GPIO.OUT)

    try:
        servo = GPIO.PWM(18, 50)
        servo.start(12.5)

        # spin left, right, then left again rather than in a continuous circle
        # to prevent the food from jamming the servo
        for index in range(0, 3):
            dutyCycle = 2.5 if (index % 2 == 0) else 12.5
            servo.ChangeDutyCycle(dutyCycle)
            time.sleep(0.8)
    finally:
        # always cleanup after ourselves
        servo.stop()
        GPIO.cleanup()

if __name__ == '__main__':
    # kick off the feeding process (move the servo)
    # we now use our new feedByGmail method to handle the feeding
    feedByGmail()

Save and exit the nano editor: CTRL+X, then Y, then Enter. As always, let’s give it a test. Send yourself an email with the subject feed cats. Drop into the python interpreter once the email arrives:

python
import CatFeeder
CatFeeder.feedByGmail()

Did it work? It did? Excellent! If it didn’t, don’t be discouraged! Python’s support community is huge, you’ll find the answer. Plus, you can always leave a comment below.

Scheduling a Cron Job to Run Our Script Regularly

We have code to move our servo motor when the correct email lands in our inbox, but it currently only runs when we physically tell it to. Let’s fix that by setting up a cron job to run our script every 60 seconds.

In order for the cron job to execute our script, we need to make our script executable. This is done by issuing the following command in the terminal: sudo chmod +x CatFeeder.py

Now we’ll add the cron job. In the terminal: crontab -e. Add the following after the very last commented line (comments start with #):

* * * * * python /path/to/your/script/CatFeeder.py

As Gabe called out in the comments, adding python after the last asterisk (*) in the line above will force the python interpreter to be used while running our script. We’re in the nano editor, so save and exit: CTRL+XYEnter. Easy as Pi (sorry). A job is now running every 60 seconds with the sole task of looking for emails and feeding cats. Don’t believe me? Give it a shot, send yourself an email with the subject feed cats.

A nicety of cron jobs is they’ll continue running automatically if your Pi ever restarts.

Scheduling an Email to be Sent Regularly

We have the code, we have the cron job. Sending an email on demand will cause the servo motor to spin, but what about spinning it on a schedule? That’s where we’ll make use of IFTTT. IFTTT stands for if this then that and allows you to connect two disparate systems based on what they call a “recipe” (trigger and action). For our purpose, we need the clock to trigger an email to be sent (action). Here’s what to do:

  1. Setup an account if you haven’t already
  2. Use IFTTT website or download the app to your phone and log in
  3. In the My Applets section, add a new applet
  4. You’ll see a screen saying if +this then that: click the +this button, then select the Date & Time service.
  5. Select the Every day at trigger, and select the time you’d like the cat feeder to activate, then hit next
  6. Now you’ll see +that, click it and find the Gmail service. You’ll need to connect the service to your Gmail account. Once finished, set the action to Send yourself an email with the subject Feed cats.
  7. Hit next, then Finish

There you have it, every day at the time you specified an email will be sent to your account. If you had any issues setting up the IFTTT recipe, check out this post for a really nice and in-depth walk-through.

Having fun? Here’s Other Ways to Send an Email

Alexa

Alexa integrates nicely with IFTTT. In the IFTTT app, create a new recipe with the trigger (+this) connecting to Alexa. You’ll need to connect the service like you did for Gmail. Once connected, set the action (+that) to send an email, like we did in the previous section.

Hands free feeding at your ready: Alexa, trigger the cat feeder.

The DO Button App

Created by the IFTTT team, the DO Button app and accompanying widget provides a straightforward way to trigger the action. You! You’re the trigger. You setup a recipe, same as before, except you’ll notice there’s no +this. You are the +this. You open the app and click the button, it then triggers an email. This app can also be configured to show on your iPhone or Androids home screen, so triggering the email is even easier.

Conclusion and Next Steps

Part one has goals and items covered, part two has Gmail automation down, and part three provided the spinning of the motor. A lot has been done so far, we’re nearly there. Up for part four? That’s where I’m headed to connect the Pi and motor to the dispenser.

Reading Gmail with Raspberry Pi and IMAPClient

Reading Gmail with Raspberry Pi and IMAPClient

In part one we covered the basics of what we’re trying to build, and what you’ll need to get started. Here in part two we’ll be playing with some python code to interact with our Gmail account using IMAPClient. Later on, part three is focused on the servo motor, while part four connects it all to the food dispenser.

The goal of this part?

  • Setup your Gmail Account with 2-step authentication for a secure way to interact with it through code. 
  • Write some code to interrogate your inbox for an email and mark it as read when found using the IMAPClient library.

Let’s get to it!

Preparing Your Gmail Account with 2-step Authentication

This is not strictly necessary, but a good security practice. If you don’t mind your credentials being in plain text in your code, feel free to skip ahead.

Since we’re going to be using email as the trigger, we’ll need a way to securely log into our Gmail account from code. You could store your credentials in plain text within the code we write, but I strongly discourage that. Instead, make use of Gmail’s two-step authentication and app passwords

I created a new Gmail account for this purpose.

  1. Log into your gmail account
  2. Navigate to the Sign-in and security page
  3. Under the Signing in to Google section, click the 2-Step Verification menu, then follow the instructions to enable 2-Step Verification
  4. Back in the Sign-in and security page right under the 2-Step Verification button you’ll see App passwords
  5. Generate a password
    1. Note: this password will be 16 digits with spaces separating every 4 digits (e.g. xxxx xxxx xxxx xxxx). Please be sure to include the spaces!

The password you generated can be used to log into your Gmail account. We don’t need it right this second, but we’ll be using it to scan for an email that demands we feed the cats!

Writing Code to Read Your Gmail

Now that we have a Gmail account ready to rock, let’s write some code to interrogate it. We’re going to be using the IMAPClient python library for this purpose, and we’ll wrap the calls to this client in our own Python class.

Install IMAPClient now from the terminal: sudo pip install imapclient

Create the GmailWrapper.py Script

Now let’s create our Gmail wrapper class: nano GmailWrapper.py. Within the GmailWrapper.py script, add the following code:

#!/usr/bin/env python

from imapclient import IMAPClient, SEEN

SEEN_FLAG = 'SEEN'
UNSEEN_FLAG = 'UNSEEN'

class GmailWrapper:
    def __init__(self, host, userName, password):
        #   force the user to pass along username and password to log in as 
        self.host = host
        self.userName = userName
        self.password = password
        self.login()

    def login(self):
        print('Logging in as ' + self.userName)
        server = IMAPClient(self.host, use_uid=True, ssl=True)
        server.login(self.userName, self.password)
        self.server = server

    #   The IMAPClient search returns a list of Id's that match the given criteria.
    #   An Id in this case identifies a specific email
    def getIdsBySubject(self, subject, unreadOnly=True, folder='INBOX'):
        #   search within the specified folder, e.g. Inbox
        self.setFolder(folder)  

        #   build the search criteria (e.g. unread emails with the given subject)
        self.searchCriteria = [UNSEEN_FLAG, 'SUBJECT', subject]

        if(unreadOnly == False):
            #   force the search to include "read" emails too
            self.searchCriteria.append(SEEN_FLAG)

        #   conduct the search and return the resulting Ids
        return self.server.search(self.searchCriteria)

    def markAsRead(self, mailIds, folder='INBOX'):
        self.setFolder(folder)
        self.server.set_flags(mailIds, [SEEN])

    def setFolder(self, folder):
        self.server.select_folder(folder)

Save and exit the nano editor: CTRL+Xthen Y, then Enter.

Verifying the GmailWrapper.py Class Works

Let’s do a test: our script is going to log into the Gmail account, search for email with a specific subject, then mark it as read. Before running the code, send yourself an email with the subject feed cats (the search is case-insensitive, by the way).

We’re going to use the Python interpreter to run our tests. In your terminal make sure you’re in the same directory as the GmailWrapper.py class we just created, then:

# press enter after each line for the interpreter to engage

# invoke the interpreter
python

# import our wrapper class for reference
from GmailWrapper import GmailWrapper

# create an instance of the class, which will also log us in
# the <password> should be the 2-step auth App Password, or your regular password
gmailWrapper = GmailWrapper('imap.gmail.com', '<your gmail username>', '<password>')

# search for any unread emails with the subject 'feed cats', and return their Ids
ids = gmailWrapper.getIdsBySubject('feed cats')

# have the interpreter print the ids variable so you know you've got something
ids

# now lets mark the email as read
gmailWrapper.markAsRead(ids)

# exit the interpreter
quit()

If everything went as planned your email should now be marked as read. Pretty neat!

Conclusion and Next Steps

Part one set the goals and outline. Now with part two wrapped up we have the ability to discern whether or not an email exists. With this power we can trigger certain events, such as spinning a servo motor. That’s exactly what we’re going to do in part three, I’ll meet you there.

Later on in part four we’ll be attaching the Pi and servo motor to the dispenser, but before we make anything permanent let’s get that servo spinning.