Log In | Users | Register

Programmer meeting November 18th, 2021

Agenda

  • New Spack Jenkinks Pipeline: PR on GitHub
    • Do we need additional testcases?
  • C2SM Spack Guidelines: IO-Page of Jonas' fork
  • Cleanup mess with c2sm_master or c2sm-master in Spack packages
  • Responsibilities C2SM? WG Day on 1 Feb: mail, discussions with groups
  • C2SM? newsletter
  • UnicodeDecodeError?

Tasks

  • Jonas and Matthieu: discuss Spack issues
  • Annika: cleanup mess with c2sm_master/*c2sm-master* in Spack packages
  • All: discuss with ICON users, which versions should be supported
  • All: discuss ICON development and version support in ICON meeting
  • Matthieu: organise containerisation meetings
  • Jonas: send mail to ask members to submit tasks. Deadline end of December
  • Matthieu: handover sandbox for int2lm bug
  • Michael: fix log file error for pgi compiler
  • Annika: create experiment for scaling analysis
  • Jonas and Michael: create standard run cases

Programmer meeting November 4th, 2021

Agenda

  • int2lm segfault on Daint
    • check additional flags when debugging
    • investigate a bit further before contacting CSCS
  • CSCS access for Annika/C2SM
  • Implement Spack Guidelines
    • put it in Spack Documentation
  • OASIS testing with Jenkins
    • TODO: Spack package, Jenkins plan
  • Funny bug when trying to open a file
  • PEP8 Github Action
  • Diskussion am Schluss: Besetzung M15.1 (Tina)

Tasks

All
  • Have joint feedback session on Python Workshop after Viz course

Jonas

  • shows Michael int2lm segfault
  • Spack documentation

Matthieu

  • Spack documentation
  • OASIS: Spack package, Jenkins plan

Tina

  • asks Mauro for node hours (less than 1000 NH / Q) / CSCS project for C2SM? members

Programmer meeting October 21st, 2021

Agenda

  • Importing DRKZ-Gitlab version of ICON into C2SM-Github
  • ICON on Euler 7
  • ICON Log file issue solved ("\varname")

Programmer meeting October 7th, 2021

Agenda

  • Task for Jonas: spend a week on the issue of Matthieu (breaking block loop in physics breaks the testsuite)
  • Michael: issue with log file format

Programmer meeting September 23rd, 2021

Agenda

  • Status on scaling analysis / benchmark proposals
  • Need for spack usage guidelines. Will be discussed at the spack meeting next week (Matthieu replacing Jonas)
  • Debug executables should be integrated in Jenkins testing
  • Status on container meeting

Actions

Matthieu
  • attend Spack meeting next week (discuss guidelines, documentation etc.)
  • check why OASIS interface broke (new snow scheme in COSMO)
Michael
  • show results of scaling analysis in next meeting
Ruth
Tina
  • Shift of python vis course (9-12 November) to 15-18 November
  • Nationaler Zukunftstag (11 November), need 1-2 persons
  • 2-3 upcoming job interviews (Katie replacement) within the next 1-2 weeks

Programmer meeting September 9th, 2021

Agenda

Actions

Matthieu
  • Support + Maintenance Meeting scheduled for 7 October 2021
  • look over support document on wiki
Michael
  • we support extensions like ICON-HAM (newest version), but without guarantee
  • look over support document on wiki
Ruth
Tina
  • Short info on "C2SM infrastructure platform" --> common support unit CSCS/C2SM

Programmer meeting August 26th, 2021

Agenda

  • Status ICON Testing on Dom
  • Status Support & Maintenance meeting
  • Compilers @Euler
  • Update configs for Spack after upgrade of Daint
  • ERA-5 Data for ICON, update and strategy

Actions

Jonas
  • send mail to technical contacts regarding next Piz Daint upgrade
Matthieu
  • set new dates (Oct+) for maintenance meeting
  • add Mauro to the meeting as person from EXCLAIM
Michael
  • update configs for Spack after upgrade of Daint
Ruth
  • ask Lukas which additional 2D variables are needed for ICON
  • copy small dataset to CSCS for Dominik to test ICONTOOLS and their ICON runs

Programmer meeting August 12th, 2021

Agenda

Actions

  • All: Send Excel sheet for user management around to groups 2x per year to check if it is up-to-date
  • Matthieu: Send out mail for support & maintenance meeting

Programmer meeting July 29th, 2021

Agenda

  • Ruth, Michael and Jonas enjoy the quiet holiday period in the office

Actions

None

Programmer meeting July 14th, 2021

Agenda

  • Upgrade Piz Daint
    • Timeline testing on Dom
    • Get in touch with Spack
  • OASIS with Spack
  • EXCLAIM user-support program
    • Get to know CSCS team (Mauro, Cerlane)

Actions

Jonas
  • Setup meeting next Monday/Tuesday with Michael/Matthieu

Jonas/Matthieu

  • Meet next week to put OASIS into Spack as soon as possible

Programmer meeting July 1st, 2021

Agenda

  • Aftermath WG-Day
    • Container for COSMO
      • CI/multistep container for Extpar
      • GPU-capable container for COSMO-ORG
    • Time constraints for discussion
      • Reserve more time for it
      • Formal approval of tasks at the end
    • Distribution of new tasks
      • More agile as previously
      • Work on tasks outside of key-competence
  • Update Timetracking

  • Dycore-Testing on Daint

Actions

Tina/Matthieu
  • Organize a taskforce or address within EXCLAIM
Matthieu
  • Take over Dycore-Testing for Daint
Michael/Matthieu
  • Create LaTeX template in overleaf for WG Day slides
Jonas
  • Organize Programmers-Meeting for July too
All
  • Organize WG-Day more in advance
  • Start 2 months in advance with task collection/planning for next WG-Day

Programmer meeting June 17th, 2021

Agenda

  • Cosmo
    • Test Dycore on Daint
    • Spack info session
    • MPI-locks in prefetching
  • ICON
    • Update 2.6.3
  • Data Management
    • Current status regarding ERA5
    • Two different purposes: analysis and icon/cosmo input
  • Daint Upgrade in September

Actions

Jonas
  • Contact Carlos for MPI-Hang
  • Put Spack session on hold, wait until more Spack users
  • Find out time effort for Dycore Testing
  • Contact Sylvaine about Spack
All
  • Inform community about upgrade of Daint at WG-Day

Programmer meeting June 3rd, 2021

Agenda

  • Lead programmer meeting
  • C2SM-Wki
    • Wiki entries of Working Groups
  • Data Management
    • Coordinate ERA5 download/storage with EMPA/ETH (Urs will attend too)
    • Several groups use ERA5 (Empa, Hymet, AtmDyn)
    • Data requirements are different for different purposes (drive a model, analyze data, etc.)
    • ERA5 will be used in the next years in C2SM community
  • C2SM slack workspace
    • Trial for a workspace for the entire C2SM community

  • Cosmo
    • Dycore testing on Daint
    • Incorporate more testcases for C2SM community in testsuite
  • Distribute Tasks Katie
    • Git workshop
    • Organize Maintenance Meeting (Matthieu)
    • Data document (Matthieu and Marie-Estelle)
    • C2SM-RCM management (Jonas)
    • data size calculator (Matthieu)
  • Preparation WG day

Actions

Jonas
  • Responsible for organization of this meeting during month June
  • C2SM-RCM management and maintencance lead (dom-testing, contact with Will)

Ruth

  • Take over coordination between different groups, contact person in groups are technical contacts

Tina

  • Dig out C2SM workspace
  • Remind Katie to make data size calculator public

Matthieu

  • Check data document from Katie
  • Data size calculator (repo still private, no one from C2SM can access it)
  • Schedule Maintencance meeting after summer break

Michael

  • Study Git course slides and think about excercises
  • Make finish in data document

Programmer meeting May 19th, 2021

Agenda

  • Katie information
  • Daint upgrade
    • Everything running
  • Spack
    • Problems with installcosmo script, can't compile dycore on Daint, works on Tsa
  • COSMO
    • Plan for pre-fetching for NetCDF to be ready for 6.0
    • Int2lm improvement: put into c2sm branch in C2SM-RCM repository
    • Need to adjust spack to compile branch
  • ICON
    • Continue to have a look at tolerance tests, make sure they are working before GPU version in place
  • Move to C2SM organization on GitHub
    • which repos
    • user group management
  • Miscellaneous
    • Hike

Actions

Jonas
  • Continue reorganization of users into group-based teams on C2SM Github organization
  • Debug spack installcosmo script with MCH
  • c2sm branch Jenkins plan weekly for C2SM-RCM COSMO and Int2lm
Jonas + Michael
  • Inform ICON users about new repository locations
Michael
  • Work on ICON tolerance tests
Matthieu
  • Add c2sm branch to int2lm repository
  • Try adding branches to spack installcosmo

Programmer meeting May 6th, 2021

Agenda

  • Creating new tolerences with jenkins_icon caused problems -> ref just created do not validate
  • Daint upgrade 5/19
    • Status COSMO [OK] and int2lm [OK]
    • Status ICON [compiles] and Icontools [OK]
    • Problem accesing input files mounted on Daint for ICON, not accesible on Dom
  • Spack
    • installcosmo and devbuildcosmo are both in final review
    • Documentation contains QuickGuide for COSMO and int2lm
    • Spack info session 2 takes place slightly after groups really started using it
      1. Start with general Spack part because people forgot content of session last year
      2. Explain COSMO-specific commands with shared screen of documentation

  • Plotting tool for ICON grids (Ideas for a solution hosted in C2SM)
    • Demand for a tool to plot pure ICON triangles
    • Integrate into PyVis course
    • Host simple solution with disclaimer,
    • Create wiki page

  • processing-chain currently used COSMO, GHG, ICON, COSMO-ART
    • Modularize the tool as much as possible to easily add new models, machine or configs
    • Matthieu first needs to make the community want such a tool

Actions

Jonas + Michael
  • Use more iterations generate references, investigate logfile of tolerance test
  • Look together into input-file problem for ICON on Dom
Jonas
  • Send email to C2SM-community, once commands and doc merged into master
  • Notify Matthieu in case Spack not ready on Monday

Matthieu

    • Push usage of Spack in groups

Programmer meeting April 22nd, 2021

Agenda

  • WG Day scheduling
    • Before summer break, 24th and 28th June, 1st, 5th, 12th, and 15th July, Tina will make a Doodle and send to members
  • Spack
    • Working config for Dom, updated config for Daint (modules up to date now)
    • Still working to find solution to add c2sm repo to MCH spack instance, doesn't work yet with spack install, but does with devbuild script
    • Important to find a solution quickly
    • Need an information session soon for C2SM COSMO users, target summer
    • Need to have consistency for C2SM users, ok if things change in background, but need to keep same workflow for users
    • Spack on Dom, needs to be reinstalled every time spack-mch changes
  • Maintenance
    • Daint Upgrade 5/19
      • Status member notification
        • People can use Foswiki functionality to get updated whenever Daint upgrade page changes, either C2SM or Will can change when new information comes in
        • Katie to send notification email to current technical contact list today
      • Testing plan on Dom
        • Jonas test COSMO compilation, Katie test technical testsuite, Michael test ICON, once COSMO and ICON working, then test Int2lm, and then DWD-ICON-TOOLS
    • Close CSCS ticket? Comment and ask for workflow to confirm when Dom is ready for testing
    • Annual meeting: time to organize next meeting, mid-end of June
  • COSMO
    • Switch cosmo-1 -> cosmo
      • Spack
        • Katie to make PR
  • ICON
    • ART: KIT informs that we can get code from DKRZ gitlab repository where all ICON versions are hosted
    • We need ANOTHER ICON-ART license, but we can distribute the code to members now
  • Miscellaneous
    • We should continue to document the support requests and ask people to use the c2sm-support email instead of contacting people directly. We should have a wiki page with a table that has a list of all the support requests.
    • We should use the time tracking software Kimai.
    • Katie to document support requests on wiki when they arrive, and then assign them. The person handling the support request then updates the table with the progress of the support.
    • It would be great to collaborate more because there is more overlap now between the work of the technical programmers. Could work together on code design, review, and testing.

Actions

Tina
  • Make Doodle for working group day
  • Follow up with missing technical contact groups

Jonas

  • Organize Spack info session
  • Test COSMO compilation on Dom once PS13 is installed

Katie

  • Send notification email to technical contact list
  • Ask Will if he can update wiki page with Daint information
  • Test COSMO on Dom
  • Comment on CSCS ticket to get workflow for finding out when Dom is ready for testing
  • Organize maintenance annual meeting
  • Follow up with missing technical contact groups

Michael

  • Test ICON on Dom once PS13 is installed

Programmer meeting April 8th, 2021

Agenda

  • Spack
    • Spack on Dom
      • Requested and received support from CSCS, needs more testing, not yet working for COSMO
      • How to proceed? Work towards Jenkins plan with cosmo-1 on Dom, open tickets to have CSCS resolve issues, lower priority
    • timeline for transition to Spack with ALL users
      • C2SM is not supporting the build scripts anymore
      • Plan an additional communication Spack session specifically for COSMO, in summer, need to communicate that build scripts are not supported, Q and A session, communicate documentation
  • Github
    • Combine organizations?
      • Move C2SM-ICON code to C2SM organization, will include CH202x code in C2SM repository, access responsibilities to be divided between Jan, Jonas, and Michael
      • Still need a contact from each group, to help manage Github and Spack communications, Katie to email all members and ask for a contact person
      • Could organize users by group (Hymet, Landclim, etc) so that it is clearer who they are
      • Should keep track of Github usernames and person name in spreadsheet to facilitate removing old users
      • Tina to add table of C2SM mailing lists to wiki
    • COSMO-GHG: Make new repo or create branch within cosmo-1?
      • COSMO-GHG and cosmo-pompa-ghg repositories in C2SM-RCM can be deleted, they are obsolete
      • Suggestion to make a fork of cosmo-1 into an EMPA organization, then put GHG code there. That way EMPA will have full control of access, etc.
    • User access management
  • Maintenance
    • Katie to chat with Will, clarification of process for upgrades
  • COSMO
  • ICON
    • ICON-ART distribution still unclear, Tina to follow up with IMK, ART license still in progress, still don't have access to ART git repository
  • General technical items
  • Community request: Use forum/wiki/Jira instead of email for support requests; change procedure, have people put topics on wiki, evaluate at certain communicated points
  • Physical group gathering (BBQ, hike, ...)

Actions

Jonas
    • Work slowly on getting COSMO running on Dom and setting up a weekly Jenkins plan
    • Spack COSMO information session in summer
    • Transfer repositories from C2SM-ICON to C2SM Github organization (with Michael)

Tina

    • Add list of C2SM mailing lists to wiki
    • Follow up with IMK about ART license

Katie

    • Email all members and ask for a contact person
    • Organize name change of cosmo-1 to cosmo

Michael

    • Remove obsolete COSMO-GHG repositories from C2SM-RCM organization

Programmer meeting March 25th, 2021

Agenda

  • Spack
    • Solution? for COSMO-package
    • Issue that summarizes challenges with dependencies and Spack in general.
    • Spack on Dom -> define procedure
  • COSMO-GHG (Empa)
    • re-organize repositories on C2SM/RCM
  • C2SM Git-repositories
    • Location for Spack-C2SM
    • Location for Processing scripts

Actions

Michael

    • Update COSMo-GHG in existing Git Repos at C2SM-RCM
    • Put COSMO-Processing chain to C2SM Github

Jonas

    • revisit Spack doc on github io pages once online
    • Work on stable solution for Cosmo builds with Spack

Programmer meeting March 11, 2021

Agenda

  • General technical items
    • Daint upgrade 3/16 Touch base with Will- status of request for C2SM notification about upgrades on Daint and access to Dom - Katie Monitor COSMO tests - Wednesday - Katie Launch ICON tests Monday + Wednesday- Jonas and Michael
    • Int2lm problems with Gnu and Cray, works with PGI, Matthieu, evaluate together with Katie if needed

  • COSMO
    • COSMO Dycore periodic boundary condition status - CSCS will do dycore development, Matthieu will port lartif_data and set up testing
  • ICON
    • Michael has access now - can start working on remaking Jenkins reference test with Jonas
    • Release in spring(?)

  • Spack
    • Dom - needs to be configured - Jonas ask best strategy for dealing with changes on Dom
    • COSMO releases- conflict with C2SM and MeteoSwiss: Carlos works on solution, switch off C2SM cosmo build, meeting Thursday, add package at that point if not resolved because solution needed rapidly
    • Contact person for Spack information from each group: senior scientist/programmer (long term): distribute information to their groups: Add to WG slides, present at WG meeting, Katie
    • devbuildcosmo builds the Dycore in Spack-stages directory instead of in cosmo repository
    • Matthieu and Jonas will work together on incorporating OASIS into spack

  • Maintenance
    • Forward to Michael (maintenance document) (done)
    • General guidelines for software we distribute: need to be finalised before EXCLAIM really gets going (end of summer, ??)

  • Any other items C2SM Organization on Github: Add Michael (done), general scripts, miscellaneous

Actions

Katie:

  • Check in with Will about upgrade notification and access to Dom
  • Monitor COSMO test Wednesday March 17
  • Add request for group contact person to WG slides

Jonas:

  • Test ICON on Daint Monday March 15 and again Wednesday March 17 (with Michael)
  • Check with CSCS about best way to maintain Spack on Dom
  • Organise next technical meeting

Tina:

  • Invite Michael to ts_ical calendar

Next meeting: March 25th, Jonas organise, add to ts_ical calendar with Zoom link, Tina invite Michael spacer

This site is managed by the Center for Climate Systems Modeling (C2SM).
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors. Ideas, requests, problems? Send feedback!