Configuring Virtualenv for Python on Windows 10 using PowerShell Wrapper

This article is based on Tyler Butler original article from 2014. Tyler has done a fanstastic job of documenting the steps, however, when  I followed the steps verbatim, I had many issues on Windows 10 machine. I have attempted to document the workarounds on Windows 10 machine to get VirtualEnv working properly on Windows 10. I have taken the liberty of using some of the steps which is already outlined in the original article.

For the sake of my learning I’m using 32 bit version of Python. This is what I noticed a lot of developers recommending to install as there is no significant advantage of using 64 bit and most libraries and drivers are available on 32 bit mode. I have not personally validated this claim but I’m not going to risk further delays with issues on 64 bit installations.

By default the latest version of Python will be on the environment path for easy execution. Here are the steps that I had to take to ensure proper installation of virtualenv so that I can use different versions of Python in parallel across projects.

  1. First ensure that you have both versions of Python installed in two different directories.
  2. I want to use Virtual environments in Python 2.7 . pip (Python Package Installer) was already installed under the Scripts folder as part of 2.7.12 version of Python.
  3. The next step is to ensure which Python version is in the PATH variable. By default when the newer versions are installed, user is provided an option to add Python executable to the PATH variable.
  4. Since we will be using a Powershell to use virtualenv, I suggest that the a proper profile is created with the right path. More info on powershell profiles can be found here.
  5. Open Powershell by Typing “Powershell” in the Search of the task bar. Right Click on it and “Run as Administrator”
  6. Ensure that Set-ExecutionPolicy is set to Unrestricted (more info on this is here.)
PS C:\>  Set-ExecutionPolicy Unrestricted

In our case, we probably want the CurrentUserAllHosts profile, since that will execute for us in every PowerShell instance. If you navigate to the location listed, there might not be a file there to edit. In that case, the following command will create a file there in the right format:

PS C:\> New-Item -Path $Profile.CurrentUserAllHosts -Type file -Force

Navigate to the profile path ($Home\Documents\WindowsPowerShell\profile.ps1). In the profile file, let’s update the PATH so that the Powershell will always invoke the Python 2.7 version by default. In addition to Python 2.7, I have added other programs that I intend to use as well. If you notice, I did not include Python 3.6.0 in the path here.

$env:Path = "C:\Program Files (x86)\Python27;C:\Program Files (x86)\Python27\Scripts;C:\Program Files (x86)\Common Files\Microsoft Shared\Windows Live;
C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Windows Live\Shared;C:\Windows\SysinternalsSuite;
C:\System Tools\SysinternalsSuite;C:\Program Files (x86)\Java\jdk1.7.0_10\bin;C:\Program Files (x86)\QuickTime\QTSystem\;C:\Program Files (x86)\Java\jre1.8.0_66;
C:\Program Files\MongoDB\Server\3.4\bin";

Adding the this PATH to powershell now, saves a whole of inexplicable errors in the subsequent steps.

Close Powershell and Launch it again as Administrator

Type get-command python on the shell and ascertain that the Source points to the Python 2.7 path

PS C:\WINDOWS\system32> get-command python
CommandType     Name          Version    Source
-----------     ----          -------    ------
Application     python.exe    0.0.0.0    C:\Program Files (x86)\Python27\python.exe

Now let’s install virtualenv

PS C:\> pip install virtualenv

Once installed, verify the virtualenv.exe file is successfully installed under the Python 2.7 Scripts Folder

Since we have already added the PATH environment variable, virtualenv.exe is available in the path. To confirm type

PS C:\WINDOWS\system32> virtualenv --version

The command will print the current version of virtualenv program and exit.

If you get an error such as:

virtualenv: The term 'virtualenv' is not recognized as the name of a cmdlet…….

This means that the PATH variable is not set properly. Verify steps 8 and 9 to ensure the PATH is set properly and includes and Scripts folder.

Now, let’s install the Powershell Wrapper for Virtualenv. More information on the Powershell Wrapper can be found here. However before installing, Create the directory structure manually under $HOME/Documents called WindowsPowerShell\Modules\VirtualEnvWrapper. This is an important step as for some reason, I kept getting “Access Denied” error when installing the PowerShell wrapper even though I had administrator rights.

PS C:\> pip install virtualenvwrapper-powershell

Once the wrapper is installed successfully, navigate to the directory where the Wrapper is installed. It should be installed under $HOME\Documents\WindowsPowerShell\Modules\VirtualEnvWrapper. Open the file  VirtualenvWrapperTabExpansion.psm1 on line 12 from this:

$_oldTabExpansion = Get-Content Function:TabExpansion
to this:
$_oldTabExpansion = Get-Content Function:TabExpansion2

This step will aviod the ” Get-Content : Cannot find path ‘Function:\TabExpansion’ because it does not exist.” error when importing the modules into PowerShell.

Once this is done, Set two additional environment variables in the system. Navigate to Contraol Panel –> System –> Advanced System Settings –>  Environment Variables.

Add the following two environment variables

Variable Name Variable Value
VIRTUALENVWRAPPER_PYTHON  <path to Python 2.7 installation>
VIRTUALENVWRAPPER_VIRTUALENV  <path to Python 2.7 installation>\Scripts

The next step is to import the wrapper modules into PowerShell. Type “Import-Module virtualenvwrapper” on the prompt

PS C:\> Import-Module virtualenvwrapper

Unfortunately, you might see an error will say something like this:

Virtualenvwrapper: Virtual environments directory
'C:\Users\seshan /.virtualenvs' does not exist. Create it or
set $env:WORKON_HOME to an existing directory.

Well, at least you know you’re on the right track! Do exactly what the message says: create the missing directory.

mkdir '~\.virtualenvs'

You might also want to change the location to store your virtual environments. To do that, set the $env:WORKON_HOME variable to wherever you want to store them. I generally stick with the default of ~\.virtualenvs though. Whatever you do, remember this location; it will come in handy.

Now before moving forward, there is a small registry update that is required to address any Registry errors such as the one shown below when running the Import-Module  Command.

Set-ItemProperty : Cannot find path 'HKCU:\Software\Python\PythonCore\2.7\InstallPath' 
because it does not exist.

To address this issues, created a file called “fix_python.reg” in a temp folder of your computer with the following information:

Windows Registry Editor Version 5.00
[HKEY_CURRENT_USER\Software\Python]
[HKEY_CURRENT_USER\Software\Python\Pythoncore]
[HKEY_CURRENT_USER\Software\Python\Pythoncore\2.7]
[HKEY_CURRENT_USER\SOFTWARE\Python\PythonCore\2.7\InstallPath]
@="C:\\Program Files (x86)\\Python27"
"ExecutablePath"="C:\\Program Files (x86)\\Python27\\python.exe"
[HKEY_CURRENT_USER\SOFTWARE\Python\PythonCore\2.7\PythonPath]
@="C:\\Program Files (x86)\\Python27\\Lib;C:\\Program Files (x86)\\Python27\\DLLs;C:\\Program Files (x86)\\Python27\\Lib\\lib-tk"

Ensure that the values are updated to reflect your correct version and path.  Execute the registry file to add it to your computer’s registry.

Once all these steps are completed, Close the Powershell and Start a new one.

In the new PowerShell, Type Import-Module virtualenvwrapper command.

PS C:\> Import-Module virtualenvwrapper

Now Type “Get-Command *virtualenv*”

PS C:\WINDOWS\system32> get-command *virtualenv*
CommandType     Name                                               Version    Source
-----------     ----                                               -------    ------
Alias     cdvirtualenv -> CDIntoVirtualEnvironment           0.1        virtualenvwrapper
Alias     cpvirtualenv -> Copy-VirtualEnvironment            0.1        virtualenvwrapper
Alias     lsvirtualenv -> Get-VirtualEnvironment             0.1        virtualenvwrapper
Alias     mkvirtualenv -> New-VirtualEnvironment             0.1        virtualenvwrapper
Alias     rmvirtualenv -> Remove-VirtualEnvironment          0.1        virtualenvwrapper
Alias     setvirtualenvproject -> Set-VirtualEnvProject      0.0        support
Function  add2virtualenv                                     0.1        virtualenvwrapper
Function  CDIntoVirtualEnvironment                           0.1        virtualenvwrapper
Function  Copy-VirtualEnvironment                            0.1        virtualenvwrapper
Function  Get-VirtualEnvironment                             0.1        virtualenvwrapper
Function  New-VirtualEnvironment                             0.1        virtualenvwrapper
Function  New-VirtualEnvProject                              0.0        support
Function  Remove-VirtualEnvironment                          0.1        virtualenvwrapper
Function  Set-VirtualEnvironment                             0.1        virtualenvwrapper
Function  Set-VirtualEnvProject                              0.0        support
Function  showvirtualenv                                     0.1        virtualenvwrapper
Function  VerifyVirtualEnv                                   0.0        support
Application virtualenv.exe                                     0.0.0.0    C:\Program Files (x86)\Python27\Scripts\virtualenv.exe

You should see an output similar to this. Ensure that the Application virtualenv.exe is also listed in the output. Please note that not all virtualenv commands have been ported successfully to the Powershell Wrapper.

Now you can create virtual environments using  New-VirtualEnvironment   cmdlet

PS C:\WINDOWS\system32> New-VirtualEnvironment testenv
New python executable in C:\Users\seshan\.virtualenvs\testenv\Scripts\python.exe
Installing setuptools, pip, wheel...done.
Added activation script por Powershell to C:\Users\seshan\.virtualenvs/testenv\Scripts.

(testenv)PS C:\WINDOWS\system32>

As you can see once the new Virtual environment is created, the shell automatically changes into the virtual environment. You can see the prefix before the drive letter with the virtual environment name.

You can also type the command workon to see the virtual environment listing and switch between them.

Now, once you exit from Powershell, All modules imported will be lost. So, when you re-invoke a new shell, you will have to run the Import-Module command again.

PS C:\> Import-Module virtualenvwrapper

Instead, remember the profile file we created in the beginning to add PATH settings, Edit that profile file and add this command below the PATH Command. Navigate to $HOME\Documents\WindowsPowerShell and Edit the profile.ps1 file and add the command as the next line below the PATH.
$env:Path = "C:\Program Files (x86)\Python27;C:\Program Files (x86)\Python27\Scripts;C:\Program Files (x86)\Common Files\Microsoft Shared\Windows Live;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Program Files (x86)\Windows Live\Shared;C:\Windows\SysinternalsSuite;C:\System Tools\SysinternalsSuite;C:\Program Files (x86)\Java\jdk1.7.0_10\bin;C:\Program Files (x86)\QuickTime\QTSystem\;C:\Program Files (x86)\Java\jre1.8.0_66;C:\Program Files\MongoDB\Server\3.4\bin";

Import-Module virtualenvwrapper

In summary, I encourage you to read Tyler’s post first and clearly understand the steps. Then follow the steps that I have outlined here to get virtualenv working on Windows 10.

OpenWorld 2011

I’m excited to be here at OpenWorld 2011 !! This is my My first OOW. Things are gearing up for the next 3 days !! There are 45,000 attendees here in Mascone Center, the largest OOW till date in Oracle’s history !

Interrel hosted a 5 hour back-to-back sessions on EPM introduction and fundamentals. I attended all the sessions and found it pretty informative (besides the obvious sales push for Oracle products!).

Attended Larry’s kick-off Keynote session last night. The event is being broadcasted live on youtube.

More to come in the following days …..you can also follow the updates on my twitter @seshanr or @openworld

How I found my iPad lost in a Taxi in NJ

I, along with my family were returning from a nice vacation in LA and had hailed a cab at Newark airport (from the Newark taxi dispatch area) yesterday evening (1/2/11). During the ride or while getting off, the iPad must have slipped from the bag in the trunk on the taxi and we did not realize it until after good 1o minutes of the taxi leaving us at our residence. The device had a bunch of files, emails and a lot of contact information that was very important to me.

The iPad did not have the ‘Mobile Me’ or any of the Tracking software that could help in finding it. Apparently, there was no way to find the iPad with just the serial number, even though Apple requires us to register the device.  So, desperate to find the missing iPad we searched several forums to find a way to track the cab that we took (I luckily happened to notice and remember the Medallion number of the cab).  After a lot of searching and calling the Newark Taxi Commission number 973-733-8912 (listed here in the official airport site) for next couple of hours in vain with no answer; we gave up for the night. Our little daughter seemed to miss it the most as she loved few of her educational games that we had installed.

The most disappointing fact was that even if the taxi driver found the device and had all intentions to return it, there was no way for him to figure out which customer it would have belonged to. The device did not have any identification markings or tags on it. I had almost lost all hopes on finding the device.

The following morning (today), I tried to call the Newark Taxi Commission again. There was no response on the listed number and not even a voicemail for me to leave a message. I think either they have an incorrect number on the site or the commission just does not answer the phone. I posted an ad in the lost and found section of craigslist and also registered the device serial number with itrackmobile site with a bleak optimism.

Finally I decided to contact the taxi dispatch center wondering if they could help. I searched for the Newark Taxi dispatch number and called the listed number (973-799-0217). The lady who picked up my call directed me to contact her supervisor (Mike Horton) at 973-332-1961 to report the loss. I called Mike, who patiently listened and took down the medallion number I had on the Taxi and said he would be contacting me as soon as he was able to locate the taxi.

Within the next 15 minutes (awesome turnaround) I received a call from Mike along with the Taxi driver.  The driver (Greg Silvester) had the iPad and as anticipated he just did not know where and whom to return it to. He offered to meet at the airport for me to pick up the device. After thanking Mike profusely for his excellent and remarkable customer service, I rushed to the airport to pick up my iPad.

Kudos and many thanks to the Newark Taxi dispatch team, especially Mike for their awesome customer service and the honesty and integrity of the NJ taxi drivers (especially, Greg Silvester) for helping in ensuring that I get my iPad back.

 

Gartner Conference

Got back from the Gartner Conference last week. This was my first track and it was an exceptional experience. The Track sessions were outstanding and they provide a radically different perspectives of technology in companies. My focus was on Enterprise Architecture and Applications. There was a major push on Cloud Computing and a very comical “roast” of Microsoft CEO.

Over all it was a very informative conference and the trip to Universal Studios was simply icing on the cake. Imagine getting through the Harry potter ride that takes 4 hours in line to be completed within 20 mins !! 🙂 and those endless river of Beers !

I’ll be providing some additional info in the upcoming days…

Implementing Hyperion Strategic Finance – Part 1

We have started work in implementing Hyperion Strategic finance. We are in the architecture phase getting the necessary hardware and identifying interfaces. Very exited about learning more about this tool.  I’ll be posting some technical implementation details on this tool very soon. So far, it seems to be a very straight forward tool from technical point of view.  I’m not a big fan of thick clients in this day and age but that’s what we have to live with when it comes to this tool.

Hopefully, We will be a web enabled version soon. I’ll be discussing more about integrating HFM and HSF here in the coming days.

Till then… Cheers..

CA Clarity SharePoint Connector

Installing SPConnector on SharePoint for accessing Clarity PPM data was an interesting learning experience. I ran into several issues with the installation and was finally able to resolve it with some manual steps.

Although the installation document is decent with the instructions, it does not cover the installation for a multi-server MOSS farm clearly. For a single server WSS implementations the documented steps may work fine.

When I ran the installation on the Central Admin Server, I got the following error (the installation log can be found in the local %temp% directory)

The EXECUTE permission was denied on the object ‘proc_getNewObjects’, database ‘SharePoint_Config’, schema ‘dbo’.

To resolve this error I had to –

  • On the DB server, Open the SQL Management Studio
  • Browse to the database in question, in our case “‘SharePoint_Config
  • Open the database and then Security > Roles > Database Roles
  • In the right part of the window, right click the WSS_Content_Application_Pools role and click Properties
  • Select the menu option “Securables”
  • Click “Add”
  • Select “Specific objects” and click “OK”
  • Click “Object Types”, select “Stored Procedures” and click “OK”
  • Add the following stored procedures: proc_getNewObjects
  • Click “OK” to add these stored procedures
  • Select the added stored procedures and select “Execute” in the “Grant” column.
  • Upon running the installation again, I got the same error for proc_putObject. Repeated the above steps for the proc_putObject stored procedure and upon re-install, I was able to get the solution installed successfully on the Central Admin Server.

    The Next step is to deploy the solution on SharePoint web-applications. I targeted the solution to specific web-apps that will be using the CA PPM web-parts.

    Once the solution is deployed, the CA PPM web-parts are ready to use. To help maximize the web-parts effectively, CA has also provided a sample SharePoint site template (stp file). This file can be uploaded into the site collections “Site Templates” gallery for end user use.

    Browsing HFM OpenLDAP using LDAP tool

    Today I ran into a scenario where I had to browse the OpenLDAP using a third party LDAP tool for a troubleshooting purpose. I found this excellent note by John Goodwin on the Oracle forums. Here’s the note from the post.

    The standard settings unless you change the native password account are :-

    Host :- <enter the hostname or ip of your openLdap machine>
    Port :- 58089
    Base :- dc=css,dc=hyperion,dc=com

    User DN :- CN=root,dc=css,dc=hyperion,dc=com
    Password :- security

    Thanks John for saving my day 🙂

    Updates

    Its been couple of months since my last update to my blog. Being a tech lead on two projects and keeping them on track takes most of your time away and what’s left has to go to family. Poor blog suffers. It has been exciting couple of months for me. I’m a tech lead on Enterprise SharePoint consolidation project and CA Clarity upgrade and re-design project at Wyndham Worldwide.

    I had been to the SharePoint 2009 Conference. It was one of the most pumped conferences I have ever attended. There has a ton of blogs and articles on SharePoint 2010 and IMHO no body does a better job at blogging about it than Joel.

    Now that the SharePoint 2010 beta bits are released, it’s time to find a box load it up with Win and SQL 2008 and start playing. I’ll write more about my learning on the beta soon (and I really hope I’d be more prompt in updating this blog.).. I’ll follow up with more on CA Clarity stuff in my next post.