Posts

Showing posts from July, 2013

Chrome Extension - Interactive Scraper

Image
This is my attempt to write an Interactive Scraper. See the Robot Icon In Browser Action Icons How to use? 1. From browser, you can choose the DOM element to scraper time to time.  Click On Capture detail 2. Data is persisted in Indexed DB of Background Page.  3. Data can be viewed or deleted by clicking "View" link from Add-In. This is implemented using Data Tables (table plug-in for jQuery ) 4. Once configured, you can run the scraper by clicking on "Roll" button. It will take you through all the URL's and collect text from DOM path and capture a screen shot for your future reference.  Development Challenges:   I have faced a few development challenges while developing this.  1. Chrome Synchronous Message Passing: I faced this problem when trying to load the URL's in a tab one after another. After the URL is loaded, the scraper should collect the text from the DOM and update it in database. There are many answers availabl

Native Vs Hybrid

There are three ways to deliver an application on Mobiles – 1. Native – Built on Native Mobile Platform 2. HTML 5 – Works in browser 3. Hybrid – Built with HTML5, Can access native features It is clear that user spend 80% of their time using Apps on mobile. That is four times more than the time they spend in browsing. Hence, browser based HTML 5 application is not the preferred way to deliver your mobile application. We have two options left before us. One, build a native application using the mobile OS platform (ex: Android or iOS). Second, use your HTML5 skills to quickly build an application and make it work as a native app on all platform using build frameworks such as PhoneGap. Many have tried to compare and contrast Native and Hybrid. In this article I will summarize the analysis I have done over this subject matter.  After going through lot of articles and presentations, a presentation from propelics appealed to me. It is unbiased and to the point. Below are the pro

Sample: Posting a Comment with a File Attachment

In SFDC Apex Code Developer, a sample to post a comment with file attachment using Chatter Apex (ConnectAPI) is given. But, the code is not copy paste-able. You may see the following error for  messageInput.messageSegments.add(textSegment); System.NullPointerException: Attempt to de-reference a null object The correct code looks as below.  String communityId = null; String feedItemId = '0D5x0000000azYd'; ConnectApi.CommentInput input = new ConnectApi.CommentInput(); ConnectApi.MessageBodyInput messageInput = new ConnectApi.MessageBodyInput(); //Added messageInput.messageSegments = new List<ConnectApi.MessageSegmentInput>(); ConnectApi.TextSegmentInput textSegment; textSegment = new ConnectApi.TextSegmentInput(); textSegment.text = 'Comment Text Body'; messageInput.messageSegments.add(textSegment); input.body = messageInput; ConnectApi.NewFileAttachmentInput attachmentInput = new ConnectApi.NewFileAttachmentInput(); attachmentInput.description

Why my custom prerequisite is saying it is corrupt?

Error: Verifying file hash Error: Setup has detected that the file '.....XXXXX.msi' has either changed since it was initially published or may be corrupt. Solution: Option 1: Uninstall .NET 4.5 and work with .NET 4.0 only Option 2:   Sign your package files with a code-signing certificate to avoid a defect introduced in .NET 4.5 References: http://stackoverflow.com/questions/13073915/clickonce-prerequisite-error-published-installer-may-be-corrupt http://msdn.microsoft.com/en-us/library/ms165429.aspx

Updating SourceData/ Data Source of the Pivot Table

Image
Errors:  1. Run time error '1004' Cannot Open PivotTable source file https://....[servlet.FileDownload?]... 2. Data source reference is not valid 3. Can not use web data source as pivot data 4.  Run-time error '-2147024809 (80070057)' Item with specified name wasn't found 5.  Error: Cannot open PivotTable source file ‘[filename[x].xls]SourceData’ 6.  Exception from HRESULT: 0x800A03EC   at System.Dynamic.ComRuntimeHelpers.CheckThrowException(Int32 hresult, ExcepInfo& excepInfo, UInt32 argErr, String message)    at CallSite.Target(Closure , CallSite , ComObject , String )    at System.Dynamic.UpdateDelegates.UpdateAndExecute2[T0,T1,TRet](CallSite site, T0 arg0, T1 arg1)    at CallSite.Target(Closure , CallSite , Object , String )    at System.Dynamic.UpdateDelegates.UpdateAndExecute2[T0,T1,TRet](CallSite site, T0 arg0, T1 arg1) Solution: We have spend a day searching around for the solution. We get this error when we open the excel sheet

Scaper Wiki : Cloud Scraping Platform

Image
Recently, I was trying to scrape some data over the web. Initially, I struggled to set up the my scraping environment based on Groovy. Soon enough, I came across Scraper Wiki, a cloud scraping platform. T hey have support for DOM Parsing and SQLLite database. And, the documentation is simple and well maintained.  Scraper Page I quickly choose PHP to scrape the data. Then, I could focus on the problem rather than worrying about setting up the platform.  Following is the one of the scripts I have written to extract JSON data into SQL Database.  1. Refereed to the existing table in ScrperWiki 2. Constructed the URL to fetch player statistics 3. Used json_decode to decode JSON data 4. Stored data in table using composite primary key (player_id,season) <?php require  'scraperwiki/simple_html_dom.php' ; ScraperWiki :: attach ( "sqads" ,  "src" ) ; $playerIds = scraperwiki :: select ( "distinct player_id from src.swdata desc" ) ; forea