[Discuss] Updating SnapFresh database

Jason Wieringa jason at wieringa.co
Fri May 2 19:43:01 PDT 2014


Hi All,

Looking back at the issue, I closed it after setting up the script; but didn’t follow through on asking about a cron job. Apologies for the confusion.

Knowing when there has been a data change on an ArcGIS Mapserver involves iterating over the excepted number id IDs and checking the ETAG in the response. It will take me a little while to derive the logic.

Cool factor, my day job focuses on making data in ArcGIS systems more accessible. Very interested if I can build an API that helps makes tasks like these simpler.

Cheers,
Jason

I’ll look into if the services I’m working on will be able to tell you if the data has changed.


On Wednesday, April 30, 2014 at 4:45 PM, Marco Abundo wrote:

> I thought it was enabled in commits related to Issue #7 ( https://github.com/ysiadf/AllIncomeFoods/issues/7 ), but I may have misunderstood.
> -m
>  
>  
>  
> On Wed, Apr 30, 2014 at 1:37 PM, Aaron Bannert <aaron at codemass.com (mailto:aaron at codemass.com)> wrote:
> > Actually the cron job was not ever enabled on the old server either.
> >  
> >  
> > On Apr 30, 2014, at 1:12 PM, Marco Abundo <mabundo at gmail.com (mailto:mabundo at gmail.com)> wrote:
> >  
> > > Thanks Aaron for all your work on this. I do wish we could re-enable the cronjob so we wouldn't be dependent on any one of us to update the data.
> > > -m  
> > >  
> > >  
> > >  
> > > On Wed, Apr 30, 2014 at 1:03 PM, Angela Shing <angela.shing at sfgov.org (mailto:angela.shing at sfgov.org)> wrote:
> > > > Thanks Aaron. I mostly want to make sure that someone remembers to update
> > > > periodically if it isn't being refreshed automatically. Also - FYI to the
> > > > group APHSA (american public human services association) has taken an
> > > > interest in SnapFresh. I'm working with them to do some publicity later
> > > > this year. Their audience/reach is human services agencies around the
> > > > country.
> > > >  
> > > >  
> > > >  
> > > > Angela Shing
> > > > Affordable Care Act Liaison
> > > > Human Services Agency
> > > > City and County of San Francisco
> > > > Desk: (415) 557-5379 (tel:%28415%29%20557-5379)
> > > >  
> > > > New Health Care Options under the Affordable Care Act
> > > > Three easy ways for San Francisco residents to learn more and enroll:
> > > > 1. Online www.CoveredCA.com (http://www.CoveredCA.com)
> > > > 2. Call Covered California (800) 300-1506 (tel:%28800%29%20300-1506). Mon - Fri 8am-5pm, Sat 8am - 6pm
> > > > 3. Visit SF Benefits Net at 1440 Harrison Street. M-F 8am-5pm
> > > >  
> > > >  
> > > >  
> > > > From:       Aaron Bannert <aaron at codemass.com (mailto:aaron at codemass.com)>
> > > > To:         SnapFresh Development Discussion <discuss at snapfresh.org (mailto:discuss at snapfresh.org)>,
> > > > Date:       04/30/2014 12:46 PM
> > > > Subject:    Re: [Discuss] Updating SnapFresh database
> > > > Sent by:    "Discuss" <discuss-bounces at snapfresh.org (mailto:discuss-bounces at snapfresh.org)>
> > > >  
> > > >  
> > > >  
> > > > I did update the data (using Jason's handy dandy script) but I didn't put a
> > > > cronjob in place yet because we might need to rethink how that works a bit,
> > > > in terms of security, error handling, bandwidth usage (we are bandwidth
> > > > constrained and that file is not small) and DB churn. Manually updating
> > > > everyone once in awhile is no problem for now too.
> > > > -aaron
> > > >  
> > > >  
> > > > On Apr 25, 2014, at 11:36 AM, Angela Shing <angela.shing at sfgov.org (mailto:angela.shing at sfgov.org)> wrote:
> > > >  
> > > >  
> > > >  
> > > >       Hey Aaron will this migrated to the new host?
> > > >       Sent from my Blackberry. Please excuse typos.
> > > >  
> > > >  
> > > >  
> > > >  
> > > >  
> > > >         From: Marco Abundo [mabundo at gmail.com (mailto:mabundo at gmail.com)]
> > > >         Sent: 04/25/2014 10:40 AM MST
> > > >         To: SnapFresh Development Discussion <discuss at snapfresh.org (mailto:discuss at snapfresh.org)>
> > > >         Subject: Re: [Discuss] Fw: Updating SnapFresh database
> > > >  
> > > >  
> > > >  
> > > >       Hey all,
> > > >  
> > > >       Jason implemented the following awhile back:
> > > >  
> > > >       https://github.com/ysiadf/AllIncomeFoods/issues/7 (Add CRON job to
> > > >       refresh data periodically)
> > > >  
> > > >       -marco
> > > >  
> > > >  
> > > >       On Thu, Apr 24, 2014 at 10:00 AM, Angela Shing <
> > > >       angela.shing at sfgov.org (mailto:angela.shing at sfgov.org)> wrote:
> > > >  
> > > >         Vakil and All
> > > >         This was the last thread I could find on auto-updating the
> > > >         SnapFresh
> > > >         database.
> > > >  
> > > >  
> > > >  
> > > >         Angela Shing
> > > >         Affordable Care Act Liaison
> > > >         Human Services Agency
> > > >         City and County of San Francisco
> > > >         Desk: (415) 557-5379 (tel:%28415%29%20557-5379)
> > > >  
> > > >         New Health Care Options under the Affordable Care Act
> > > >         Three easy ways for San Francisco residents to learn more and
> > > >         enroll:
> > > >         1. Online www.CoveredCA.com (http://www.CoveredCA.com)
> > > >         2. Call Covered California (800) 300-1506 (tel:%28800%29%20300-1506). Mon - Fri 8am-5pm, Sat
> > > >         8am - 6pm
> > > >         3. Visit SF Benefits Net at 1440 Harrison Street. M-F 8am-5pm
> > > >         ----- Forwarded by Angela Shing/DHS/CCSF on 04/24/2014 09:59 AM
> > > >         -----
> > > >  
> > > >         From:       Marco Abundo <mabundo at gmail.com (mailto:mabundo at gmail.com)>
> > > >         To:         SnapFresh Development Discussion <discuss at snapfresh.org (mailto:discuss at snapfresh.org)
> > > >         >,
> > > >         Date:       12/04/2012 07:30 PM
> > > >         Subject:    Re: [Discuss] Updating SnapFresh database
> > > >         Sent by:    discuss-bounces at snapfresh.org (mailto:discuss-bounces at snapfresh.org)
> > > >  
> > > >  
> > > >  
> > > >         Excellent! Thanks Michelle.
> > > >         -m
> > > >  
> > > >         Sent from my iPhone
> > > >  
> > > >         On Dec 4, 2012, at 7:16 PM, Michelle Koeth <
> > > >         michelle at codeforamerica.org (mailto:michelle at codeforamerica.org)>
> > > >         wrote:
> > > >  
> > > >         > Hey All -
> > > >         >
> > > >         > Marco and I are still working the API issue, but in the meantime,
> > > >         I
> > > >         > wrote a rake task that refreshes the database, and it
> > > >         successfully ran
> > > >         > tonight on the production server, so the data is refreshed. Feel
> > > >         free
> > > >         > to check it out/test it against anything new that you believe
> > > >         should
> > > >         > be in there.
> > > >         >
> > > >         > Cheers
> > > >         > Michelle
> > > >         >
> > > >         > On Fri, Nov 30, 2012 at 9:11 PM, Angela Shing <
> > > >         angela.shing at sfgov.org (mailto:angela.shing at sfgov.org)>
> > > >         wrote:
> > > >         >>
> > > >         >> Yay! THANKS!!!!!
> > > >         >>
> > > >         >>
> > > >         >>
> > > >         >> Sent with Good (www.good.com (http://www.good.com))
> > > >         >>
> > > >         >> Angela Shing
> > > >         >> CalWIN Manager
> > > >         >> SFHSA
> > > >         >> angela.shing at sfgov.org (mailto:angela.shing at sfgov.org)
> > > >         >>
> > > >         >>
> > > >         >> -------- Original Message --------
> > > >         >>
> > > >         >> From :      discuss-bounces at snapfresh.org (mailto:discuss-bounces at snapfresh.org)
> > > >         >> To :         SnapFresh Development Discussion <
> > > >         discuss at snapfresh.org (mailto:discuss at snapfresh.org)>
> > > >         >> Cc :
> > > >         >> Sent on : 11/30 06:02:05 PM PST
> > > >         >> Subject : Re: [Discuss] Updating SnapFresh database
> > > >         >>
> > > >         >> Awesome. Thanks!
> > > >         >>
> > > >         >> On Fri, Nov 30, 2012 at 9:00 PM, Marco Abundo <mabundo at gmail.com (mailto:mabundo at gmail.com)
> > > >         > wrote:
> > > >         >>> Michelle, it's not on there yet. I'll create a new branch for
> > > >         that and
> > > >         >>> push to github, hopefully this weekend.
> > > >         >>>
> > > >         >>> On Fri, Nov 30, 2012 at 5:52 PM, Michelle Koeth
> > > >         >>> <michelle at codeforamerica.org (mailto:michelle at codeforamerica.org)> wrote:
> > > >         >>>> * maybe we don't have extra fields right now, but what I mean
> > > >         to say
> > > >         >>>> is that going forward I think we should consider being
> > > >         custodians of
> > > >         >>>> the "extra" stuff and so we will still have our own database.
> > > >         But for
> > > >         >>>> all FNS fields we can just pull from their API.
> > > >         >>>>
> > > >         >>>> On Fri, Nov 30, 2012 at 8:50 PM, Michelle Koeth
> > > >         >>>> <michelle at codeforamerica.org (mailto:michelle at codeforamerica.org)> wrote:
> > > >         >>>>> I think we're going to have to do both anyways since we have
> > > >         extra
> > > >         >>>>> fields that FNS does not. I think this is the way to go.
> > > >         >>>>>
> > > >         >>>>> Marco, is your working prototype of the FNS API code on our
> > > >         github
> > > >         >> repo?
> > > >         >>>>>
> > > >         >>>>> Angela reminded me why I avoided coding for the FNS API in
> > > >         the first
> > > >         >>>>> place: SOAP (shiver). Is it really a SOAP interface?? Anyhow,
> > > >         Marco,
> > > >         >>>>> if you have the prototype - awesome. Let me know where it is
> > > >         and lets
> > > >         >>>>> work together to get it implemented.
> > > >         >>>>>
> > > >         >>>>> Cheers
> > > >         >>>>> Michelle
> > > >         >>>>>
> > > >         >>>>> On Fri, Nov 30, 2012 at 8:37 PM, Aaron Bannert <
> > > >         aaron at codemass.com (mailto:aaron at codemass.com)>
> > > >         >> wrote:
> > > >         >>>>>> Someone could theoretically have our app code call their
> > > >         REST API
> > > >         when
> > > >         >> needed, so you'd get the best of both worlds (our biz logic and
> > > >         their
> > > >         >> always-up-to-date DB)
> > > >         >>>>>> -aaron
> > > >         >>>>>>
> > > >         >>>>>>
> > > >         >>>>>> On Nov 30, 2012, at 5:28 PM, Marco Abundo <mabundo at gmail.com (mailto:mabundo at gmail.com)
> > > >         > wrote:
> > > >         >>>>>>
> > > >         >>>>>>> Angela, I actually coded up a prototype version of the app
> > > >         that
> > > >         >>>>>>> consumes SNAP retailer locations from FNS' REST API which
> > > >         you
> > > >         >>>>>>> mentioned below.
> > > >         >>>>>>>
> > > >         >>>>>>> However, that would bypass our SnapFresh database and
> > > >         business
> > > >         logic
> > > >         >>>>>>> (listing healthier retailers first). So I did not proceed
> > > >         much
> > > >         beyond
> > > >         >>>>>>> that, pending group input.
> > > >         >>>>>>>
> > > >         >>>>>>> -m
> > > >         >>>>>>>
> > > >         >>>>>>> On Fri, Nov 30, 2012 at 5:07 PM, Angela Shing
> > > >         >> <angela.shing at sfgov.org (mailto:angela.shing at sfgov.org)> wrote:
> > > >         >>>>>>>> Hi All
> > > >         >>>>>>>> With the increasing usage of SnapFresh I think it's
> > > >         getting more
> > > >         and
> > > >         >> more
> > > >         >>>>>>>> critical to update the SnapFresh database. I believe it
> > > >         was loaded
> > > >         >> with the
> > > >         >>>>>>>> FNS retailer data in July 2011? At a minimum I'd like to
> > > >         have it
> > > >         >> updated
> > > >         >>>>>>>> with more recent data. But after speaking with FNS - they
> > > >         said
> > > >         that
> > > >         >> there
> > > >         >>>>>>>> is an open interface so we could hook into it for real
> > > >         time
> > > >         updates.
> > > >         >>>>>>>> Supported interfaces are REST SOAP
> > > >         >>>>>>>> (http://www.snapretailerlocator.com/ArcGIS/rest/services)
> > > >         >>>>>>>>
> > > >         >>>>>>>> I've lost track of who from the original team is on this
> > > >         >> distribution list,
> > > >         >>>>>>>> but do we have anyone who can either update the database?
> > > >         or
> > > >         better
> > > >         >> -
> > > >         >>>>>>>> change the configuration so we can hook into the real time
> > > >         updates
> > > >         >> from the
> > > >         >>>>>>>> FNS database?
> > > >         >>>>>>>>
> > > >         >>>>>>>>
> > > >         >>>>>>>>
> > > >         >>>>>>>> Angela Shing
> > > >         >>>>>>>> CalWIN Manager
> > > >         >>>>>>>> Human Services Agency
> > > >         >>>>>>>> City and County of San Francisco
> > > >         >>>>>>>> Desk: (415) 557-5881 (tel:%28415%29%20557-5881)
> > > >         >>>>>>>> email: angela.shing at sfgov.org (mailto:angela.shing at sfgov.org)
> > > >         >>>>>>>>
> > > >         >>>>>>>> _______________________________________________
> > > >         >>>>>>>> Discuss mailing list
> > > >         >>>>>>>> Discuss at snapfresh.org (mailto:Discuss at snapfresh.org)
> > > >         >>>>>>>> http://www.snapfresh.org/mailman/listinfo/discuss
> > > >         >>>>>>> _______________________________________________
> > > >         >>>>>>> Discuss mailing list
> > > >         >>>>>>> Discuss at snapfresh.org (mailto:Discuss at snapfresh.org)
> > > >         >>>>>>> http://www.snapfresh.org/mailman/listinfo/discuss
> > > >         >>>>>>
> > > >         >>>>>> _______________________________________________
> > > >         >>>>>> Discuss mailing list
> > > >         >>>>>> Discuss at snapfresh.org (mailto:Discuss at snapfresh.org)
> > > >         >>>>>> http://www.snapfresh.org/mailman/listinfo/discuss
> > > >         >>>>>
> > > >         >>>>>
> > > >         >>>>>
> > > >         >>>>> --
> > > >         >>>>> Michelle M. Koeth
> > > >         >>>>> Code for America Fellow Alumna
> > > >         >>>>> http://www.codeforamerica.org
> > > >         >>>>
> > > >         >>>>
> > > >         >>>>
> > > >         >>>> --
> > > >         >>>> Michelle M. Koeth
> > > >         >>>> Code for America Fellow Alumna
> > > >         >>>> http://www.codeforamerica.org
> > > >         >>>> _______________________________________________
> > > >         >>>> Discuss mailing list
> > > >         >>>> Discuss at snapfresh.org (mailto:Discuss at snapfresh.org)
> > > >         >>>> http://www.snapfresh.org/mailman/listinfo/discuss
> > > >         >>> _______________________________________________
> > > >         >>> Discuss mailing list
> > > >         >>> Discuss at snapfresh.org (mailto:Discuss at snapfresh.org)
> > > >         >>> http://www.snapfresh.org/mailman/listinfo/discuss
> > > >         >>
> > > >         >>
> > > >         >>
> > > >         >> --
> > > >         >> Michelle M. Koeth
> > > >         >> Code for America Fellow Alumna
> > > >         >> http://www.codeforamerica.org
> > > >         >> _______________________________________________
> > > >         >> Discuss mailing list
> > > >         >> Discuss at snapfresh.org (mailto:Discuss at snapfresh.org)
> > > >         >> http://www.snapfresh.org/mailman/listinfo/discuss
> > > >         >>
> > > >         >> _______________________________________________
> > > >         >> Discuss mailing list
> > > >         >> Discuss at snapfresh.org (mailto:Discuss at snapfresh.org)
> > > >         >> http://www.snapfresh.org/mailman/listinfo/discuss
> > > >         >
> > > >         >
> > > >         >
> > > >         > --
> > > >         > Michelle M. Koeth
> > > >         > Code for America Fellow Alumna
> > > >         > http://www.codeforamerica.org
> > > >         > _______________________________________________
> > > >         > Discuss mailing list
> > > >         > Discuss at snapfresh.org (mailto:Discuss at snapfresh.org)
> > > >         > http://www.snapfresh.org/mailman/listinfo/discuss
> > > >         _______________________________________________
> > > >         Discuss mailing list
> > > >         Discuss at snapfresh.org (mailto:Discuss at snapfresh.org)
> > > >         http://www.snapfresh.org/mailman/listinfo/discuss
> > > >  
> > > >         _______________________________________________
> > > >         Discuss mailing list
> > > >         Discuss at snapfresh.org (mailto:Discuss at snapfresh.org)
> > > >         http://www.snapfresh.org/mailman/listinfo/discuss
> > > >  
> > > >       _______________________________________________
> > > >       Discuss mailing list
> > > >       Discuss at snapfresh.org (mailto:Discuss at snapfresh.org)
> > > >       http://www.snapfresh.org/mailman/listinfo/discuss
> > > >       _______________________________________________
> > > >       Discuss mailing list
> > > >       Discuss at snapfresh.org (mailto:Discuss at snapfresh.org)
> > > >       http://www.snapfresh.org/mailman/listinfo/discuss
> > > > _______________________________________________
> > > > Discuss mailing list
> > > > Discuss at snapfresh.org (mailto:Discuss at snapfresh.org)
> > > > http://www.snapfresh.org/mailman/listinfo/discuss
> > > >  
> > > >  
> > > > _______________________________________________
> > > > Discuss mailing list
> > > > Discuss at snapfresh.org (mailto:Discuss at snapfresh.org)
> > > > http://www.snapfresh.org/mailman/listinfo/discuss
> > >  
> > > _______________________________________________
> > > Discuss mailing list
> > > Discuss at snapfresh.org (mailto:Discuss at snapfresh.org)
> > > http://www.snapfresh.org/mailman/listinfo/discuss
> >  
> > _______________________________________________
> > Discuss mailing list
> > Discuss at snapfresh.org (mailto:Discuss at snapfresh.org)
> > http://www.snapfresh.org/mailman/listinfo/discuss
> >  
>  
> _______________________________________________
> Discuss mailing list
> Discuss at snapfresh.org (mailto:Discuss at snapfresh.org)
> http://www.snapfresh.org/mailman/listinfo/discuss
>  
>  


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.snapfresh.org/pipermail/discuss/attachments/20140502/0c63f255/attachment-0001.html>


More information about the Discuss mailing list