Skip to content

Commit

Permalink
CV model update (2.17.0) (#723)
Browse files Browse the repository at this point in the history
* Use cv model 2.13, requires newer plugin

* Bump version and build

* Use cv model 2.13, requires newer plugin

* Bump version and build

* Use vision-plugin with speedup by taxonomy cutoff

* Bump build

* Bundle update

* Redo bundle with deleted Gemfile.lock

* Update plugin. Add option to set cutoff from JS

* Cycle through different taxonomy cutoffs

* Bump build

* Update vision plugin

* Change cycle back to one cutoff value only

* Bump build

* Show correct about text, and not missing translation

Closes #713

* Use history for going back in navigation

Closes #715
Closes #675

* Bump build

* Update vision-plugin

* Add a bit of padding to bottom of challenges screen

* Bump build

* Update with stable cv plugin

* Log timeElapsed for cv on native side

* Bump build

* Comment

* Update README.md

* Use stable vc plugin main

Technically this means we now support geomodel on iOS. We would need to add it to the app an connect to the frame processor though.

* Bump build

* Update vision-plugin

* Bump build

* Remove taxonomy cutoff override

Now it uses the plugin's inbuilt taxonomy cutoff of 0.001 the top cv score.

* Remove timesRun state

* Bump build

* Card that announces the new cv model on the home screen (#717)

* Fix vision plugin to version commit on main there

* Update Podfile.lock

* Basic version of the updates card

* Count how many species are in cv 2.13

* Looks nicer

* Bump build

* Revert back to storing only five results at a time

* Bump build

* Update vision-plugin to use new names for options

No change in Seek required

* Hardcode to never show a specific taxon

* Bump build

* Update package.json

* Remove deprecated model files; Use model on iOS directly from bundle

* Bump build

* Update fastlane
  • Loading branch information
jtklein authored Jan 8, 2025
1 parent a4859df commit 6add69f
Show file tree
Hide file tree
Showing 18 changed files with 159 additions and 55 deletions.
8 changes: 4 additions & 4 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ GEM
artifactory (3.0.17)
atomos (0.1.3)
aws-eventstream (1.3.0)
aws-partitions (1.1023.0)
aws-partitions (1.1026.0)
aws-sdk-core (3.214.0)
aws-eventstream (~> 1, >= 1.3.0)
aws-partitions (~> 1, >= 1.992.0)
Expand Down Expand Up @@ -107,8 +107,8 @@ GEM
faraday-em_synchrony (1.0.0)
faraday-excon (1.1.0)
faraday-httpclient (1.0.1)
faraday-multipart (1.0.4)
multipart-post (~> 2)
faraday-multipart (1.1.0)
multipart-post (~> 2.0)
faraday-net_http (1.0.2)
faraday-net_http_persistent (1.2.0)
faraday-patron (1.0.0)
Expand Down Expand Up @@ -209,7 +209,7 @@ GEM
i18n (1.14.6)
concurrent-ruby (~> 1.0)
jmespath (1.6.2)
json (2.9.0)
json (2.9.1)
jwt (2.9.3)
base64
mini_magick (4.13.2)
Expand Down
4 changes: 2 additions & 2 deletions android/app/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -98,8 +98,8 @@ android {
applicationId "org.inaturalist.seek"
minSdkVersion rootProject.ext.minSdkVersion
targetSdkVersion rootProject.ext.targetSdkVersion
versionCode 377
versionName "2.16.4"
versionCode 376
versionName "2.17.0"
// for creating ic_seek_adaptiveappicon.xml
vectorDrawables.useSupportLibrary = true
}
Expand Down
2 changes: 1 addition & 1 deletion android/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ buildscript {
ndkVersion = "25.1.8937393"
kotlinVersion = "1.8.0"
// This specifies which tensorflow-lite version to use for our vision-plugin.
tensorflowVersion = "2.1.0"
tensorflowVersion = "2.14.0"
}
repositories {
google()
Expand Down
4 changes: 3 additions & 1 deletion components/Camera/ARCamera/ARCamera.js
Original file line number Diff line number Diff line change
Expand Up @@ -227,7 +227,9 @@ const ARCamera = ( ): Node => {

// not looking at kingdom or phylum as we are currently not displaying results for those ranks
const wantedRanks = ["species", "genus", "family", "order", "class"];
const wantedPredictions = predictions.filter( p => wantedRanks.includes( p.rank ) );
let wantedPredictions = predictions.filter( p => wantedRanks.includes( p.rank ) );
const unwantedTaxa = [1044608, 1044607, 973699, 152504, 1128037];
wantedPredictions = wantedPredictions.filter( p => !unwantedTaxa.includes( p.taxon_id ) );

dispatch( { type: "SET_PREDICTIONS", predictions: wantedPredictions } );

Expand Down
3 changes: 2 additions & 1 deletion components/Camera/ARCamera/FrameProcessorCamera.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,7 @@ const FrameProcessorCamera = ( props: Props ) => {
const fps = 1;
const handleResult = Worklets.createRunOnJS( ( result: InatVision.Result, timeTaken: number ) => {
setLastTimestamp( result.timestamp );
console.log( "result.timeElapsed", result.timeElapsed );
framesProcessingTime.push( timeTaken );
if ( framesProcessingTime.length >= 10 ) {
const avgTime = framesProcessingTime.reduce( ( a, b ) => a + b, 0 ) / 10;
Expand Down Expand Up @@ -217,7 +218,7 @@ const FrameProcessorCamera = ( props: Props ) => {
try {
const timeBefore = Date.now();
const result = InatVision.inatVision( frame, {
version: "1.0",
version: "2.13",
modelPath,
taxonomyPath,
confidenceThreshold,
Expand Down
2 changes: 2 additions & 0 deletions components/Home/HomeScreen.js
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ import SpeciesNearby from "./SpeciesNearby/SpeciesNearby";
import Announcements from "./Announcements/Announcements";
import GetStarted from "../Modals/GetStarted";
import ChallengeCard from "./Challenges/ChallengeCard";
import Updates from "./Updates/Updates";
import { checkIfCardShown } from "../../utility/helpers";
import RNModal from "../UIComponents/Modals/Modal";
import ScrollNoHeader from "../UIComponents/Screens/ScrollNoHeader";
Expand Down Expand Up @@ -112,6 +113,7 @@ const HomeScreen = ( ): Node => {
<Announcements />
<SeekYearInReviewCard />
<ChallengeCard />
<Updates />
<INatCard />
<DonateCard />
</ScrollNoHeader>
Expand Down
30 changes: 30 additions & 0 deletions components/Home/Updates/Updates.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
import React from "react";
import { View, Image } from "react-native";

import i18n from "../../../i18n";
import icons from "../../../assets/icons";
import { baseTextStyles } from "../../../styles/textStyles";
import { viewStyles, imageStyles, textStyles } from "../../../styles/home/updates";
import GreenText from "../../UIComponents/GreenText";
import StyledText from "../../UIComponents/StyledText";

const Updates = ( ) => (
<View style={viewStyles.container}>
<View style={viewStyles.header}>
<GreenText text="updates_card.header" />
</View>
<View style={[viewStyles.row, viewStyles.center]}>
<Image source={icons.cameraGreen} style={imageStyles.image} />
<View>
<StyledText style={[baseTextStyles.emptyState, textStyles.textWidth]}>
{i18n.t( "updates_card.updated_id_model" )}
</StyledText>
<StyledText style={[baseTextStyles.body, textStyles.text, textStyles.textWidth]}>
{i18n.t( "updates_card.over_x_species" )}
</StyledText>
</View>
</View>
</View>
);

export default Updates;
10 changes: 5 additions & 5 deletions fastlane/Fastfile
Original file line number Diff line number Diff line change
Expand Up @@ -22,28 +22,28 @@ ANDROID_CAMERA_PATH = File.join( "..", "android", "app", "src", "main", "assets"
IOS_CAMERA_PATH = File.join( "..", "ios" )

def android_taxonomy_present?
path = File.join( ANDROID_CAMERA_PATH, "taxonomy_v1.csv" )
path = File.join( ANDROID_CAMERA_PATH, "taxonomy_v2_13.csv" )
unless File.exist?( path )
UI.abort_with_message! "Android production taxonomy file is missing at #{path}"
end
end

def ios_taxonomy_present?
path = File.join( IOS_CAMERA_PATH, "taxonomy_v1.json" )
path = File.join( IOS_CAMERA_PATH, "taxonomy_v2_13.json" )
unless File.exist?( path )
UI.abort_with_message! "iOS production taxonomy file is missing at #{path}"
end
end

def android_model_present?
path = File.join( ANDROID_CAMERA_PATH, "optimized_model_v1.tflite" )
path = File.join( ANDROID_CAMERA_PATH, "optimized_model_v2_13.tflite" )
unless File.exist?( path )
UI.abort_with_message! "Android production vision model file is missing at #{path}"
end
end

def ios_model_present?
path = File.join( IOS_CAMERA_PATH, "optimized_model_v1.mlmodel" )
path = File.join( IOS_CAMERA_PATH, "optimized_model_v2_13.mlmodel" )
unless File.exist?( path )
UI.abort_with_message! "iOS production vision model file is missing at #{path}"
end
Expand Down Expand Up @@ -141,7 +141,7 @@ platform :android do
)
end

desc "Distributes an Android bundle to the Play Store beta track"
desc "Distributes an Android bundle to the Play Store internal beta track"
lane :internal do
required_android_files_present?
validate_play_store_json_key(json_key: ENV["JSON_KEY_ANDROID"])
Expand Down
2 changes: 1 addition & 1 deletion fastlane/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ Upload description to Google Play
[bundle exec] fastlane android internal
```

Distributes an Android bundle to the Play Store beta track
Distributes an Android bundle to the Play Store internal beta track

### android beta

Expand Down
4 changes: 2 additions & 2 deletions ios/Podfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -1229,7 +1229,7 @@ PODS:
- VisionCamera/React (4.0.5):
- React-Core
- VisionCamera/FrameProcessors
- VisionCameraPluginInatVision (4.0.4):
- VisionCameraPluginInatVision (4.1.4):
- React-Core
- Yoga (1.14.0)

Expand Down Expand Up @@ -1649,7 +1649,7 @@ SPEC CHECKSUMS:
SDWebImageWebPCoder: e38c0a70396191361d60c092933e22c20d5b1380
SocketRocket: f32cd54efbe0f095c4d7594881e52619cfe80b17
VisionCamera: 4c1d19f1ac09f2f42f758e306fcf642536627357
VisionCameraPluginInatVision: 547e06ae37d5794992c17fc93ffa9c5363ccda42
VisionCameraPluginInatVision: 1bec4436cc2d2a952435ddbce2ee5457faa3ac20
Yoga: 2a16e58450c48e110211dae1159fb114bbcdcfc0

PODFILE CHECKSUM: 10e968d6ad2bca83e521f2b5695a4b3ffd1e8753
Expand Down
16 changes: 8 additions & 8 deletions ios/Seek.xcodeproj/project.pbxproj
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@
13B07FBF1A68108700A75B9A /* Images.xcassets in Resources */ = {isa = PBXBuildFile; fileRef = 13B07FB51A68108700A75B9A /* Images.xcassets */; };
13B07FC11A68108700A75B9A /* main.m in Sources */ = {isa = PBXBuildFile; fileRef = 13B07FB71A68108700A75B9A /* main.m */; };
3750FC0326CC35EF00C6F1DA /* Launch Screen.storyboard in Resources */ = {isa = PBXBuildFile; fileRef = 3750FC0226CC35EF00C6F1DA /* Launch Screen.storyboard */; };
377216512880BBF300B00213 /* optimized_model_v1.mlmodel in Sources */ = {isa = PBXBuildFile; fileRef = 3772164F2880BBF300B00213 /* optimized_model_v1.mlmodel */; };
377216522880BBF300B00213 /* taxonomy_v1.json in Resources */ = {isa = PBXBuildFile; fileRef = 377216502880BBF300B00213 /* taxonomy_v1.json */; };
377216512880BBF300B00213 /* optimized_model_v2_13.mlmodel in Sources */ = {isa = PBXBuildFile; fileRef = 3772164F2880BBF300B00213 /* optimized_model_v2_13.mlmodel */; };
377216522880BBF300B00213 /* taxonomy_v2_13.json in Resources */ = {isa = PBXBuildFile; fileRef = 377216502880BBF300B00213 /* taxonomy_v2_13.json */; };
378785D7244E5B7C00D4509C /* File.swift in Sources */ = {isa = PBXBuildFile; fileRef = 378785D6244E5B7C00D4509C /* File.swift */; };
37D71CD4233BE712003423F8 /* CloudKit.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = 37D71CD3233BE712003423F8 /* CloudKit.framework */; };
37ED91EF22A05D6B00BF47F4 /* JavaScriptCore.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = 37ED91AD22A05D6B00BF47F4 /* JavaScriptCore.framework */; };
Expand Down Expand Up @@ -60,8 +60,8 @@
15D53123BCFE4D5992B467B6 /* libRNWebGL.a */ = {isa = PBXFileReference; explicitFileType = undefined; fileEncoding = 9; includeInIndex = 0; lastKnownFileType = archive.ar; path = libRNWebGL.a; sourceTree = "<group>"; };
3750FC0226CC35EF00C6F1DA /* Launch Screen.storyboard */ = {isa = PBXFileReference; lastKnownFileType = file.storyboard; path = "Launch Screen.storyboard"; sourceTree = "<group>"; };
375C595B24491334000C16B5 /* Seek-Bridging-Header.h */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.c.h; path = "Seek-Bridging-Header.h"; sourceTree = "<group>"; };
3772164F2880BBF300B00213 /* optimized_model_v1.mlmodel */ = {isa = PBXFileReference; lastKnownFileType = file.mlmodel; path = optimized_model_v1.mlmodel; sourceTree = "<group>"; };
377216502880BBF300B00213 /* taxonomy_v1.json */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.json; path = taxonomy_v1.json; sourceTree = "<group>"; };
3772164F2880BBF300B00213 /* optimized_model_v2_13.mlmodel */ = {isa = PBXFileReference; lastKnownFileType = file.mlmodel; path = optimized_model_v2_13.mlmodel; sourceTree = "<group>"; };
377216502880BBF300B00213 /* taxonomy_v2_13.json */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text.json; path = taxonomy_v2_13.json; sourceTree = "<group>"; };
378785D6244E5B7C00D4509C /* File.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = File.swift; sourceTree = "<group>"; };
379A071821C4328C00F176E3 /* Seek.entitlements */ = {isa = PBXFileReference; lastKnownFileType = text.plist.entitlements; name = Seek.entitlements; path = Seek/Seek.entitlements; sourceTree = "<group>"; };
37D71CD3233BE712003423F8 /* CloudKit.framework */ = {isa = PBXFileReference; lastKnownFileType = wrapper.framework; name = CloudKit.framework; path = System/Library/Frameworks/CloudKit.framework; sourceTree = SDKROOT; };
Expand Down Expand Up @@ -228,8 +228,8 @@
8F91A11C2BD066F500513552 /* Lato-Medium.ttf */,
8F91A1162BD066F500513552 /* Lato-MediumItalic.ttf */,
8F91A1192BD066F500513552 /* Lato-Regular.ttf */,
3772164F2880BBF300B00213 /* optimized_model_v1.mlmodel */,
377216502880BBF300B00213 /* taxonomy_v1.json */,
3772164F2880BBF300B00213 /* optimized_model_v2_13.mlmodel */,
377216502880BBF300B00213 /* taxonomy_v2_13.json */,
);
name = Resources;
sourceTree = "<group>";
Expand Down Expand Up @@ -360,7 +360,7 @@
13B07FBF1A68108700A75B9A /* Images.xcassets in Resources */,
8F91A1232BD066F500513552 /* Lato-Italic.ttf in Resources */,
8F1AC67B2BC1B6AD002F994B /* PrivacyInfo.xcprivacy in Resources */,
377216522880BBF300B00213 /* taxonomy_v1.json in Resources */,
377216522880BBF300B00213 /* taxonomy_v2_13.json in Resources */,
8F91A1222BD066F500513552 /* Lato-BoldItalic.ttf in Resources */,
8F91A1212BD066F500513552 /* Lato-Regular.ttf in Resources */,
8F91A1202BD066F500513552 /* Lato-Bold.ttf in Resources */,
Expand Down Expand Up @@ -467,7 +467,7 @@
isa = PBXSourcesBuildPhase;
buildActionMask = 2147483647;
files = (
377216512880BBF300B00213 /* optimized_model_v1.mlmodel in Sources */,
377216512880BBF300B00213 /* optimized_model_v2_13.mlmodel in Sources */,
378785D7244E5B7C00D4509C /* File.swift in Sources */,
13B07FBC1A68108700A75B9A /* AppDelegate.mm in Sources */,
13B07FC11A68108700A75B9A /* main.m in Sources */,
Expand Down
4 changes: 2 additions & 2 deletions ios/Seek/Info.plist
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,11 @@
<key>CFBundlePackageType</key>
<string>APPL</string>
<key>CFBundleShortVersionString</key>
<string>2.16.5</string>
<string>2.17.0</string>
<key>CFBundleSignature</key>
<string>????</string>
<key>CFBundleVersion</key>
<string>377</string>
<string>376</string>
<key>ITSAppUsesNonExemptEncryption</key>
<false />
<key>LSApplicationQueriesSchemes</key>
Expand Down
16 changes: 9 additions & 7 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 3 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "seek",
"version": "2.16.5",
"version": "2.17.0",
"description": "Seek by iNaturalist repo",
"private": true,
"scripts": {
Expand All @@ -10,7 +10,7 @@
"ios:release": "npx react-native run-ios --device --mode Release",
"start": "npx react-native start",
"clean": "./scripts/clean.sh",
"clean-start": "npx react-native clean-project-auto && npx pod-install && npm start -- --reset-cache",
"clean-start": "npx react-native clean-project-auto && npx pod-install && npm run clean && npm start -- --reset-cache",
"lint": "npm run lint:eslint",
"lint:eslint": "eslint .",
"lint:flow": "flow check",
Expand Down Expand Up @@ -89,7 +89,7 @@
"react-native-webview": "^11.26.1",
"react-native-worklets-core": "1.3.3",
"realm": "^12.5.1",
"vision-camera-plugin-inatvision": "github:inaturalist/vision-camera-plugin-inatvision"
"vision-camera-plugin-inatvision": "github:inaturalist/vision-camera-plugin-inatvision#8788e6d6718a4501056bad1f9ee5dbcfd354be92"
},
"devDependencies": {
"@babel/core": "^7.20.0",
Expand Down
39 changes: 39 additions & 0 deletions styles/home/updates.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
import { StyleSheet } from "react-native";
import { center, colors, row } from "../global";

const viewStyles = StyleSheet.create( {
center,
container: {
backgroundColor: colors.white
},
header: {
paddingBottom: 21,
paddingHorizontal: 22,
paddingTop: 25
},
row
} );

const imageStyles = StyleSheet.create( {
image: {
height: 68,
marginRight: 28,
resizeMode: "contain",
width: 68
}
} );

const textStyles = StyleSheet.create( {
text: {
marginTop: 11
},
textWidth: {
width: 215
}
} );

export {
viewStyles,
imageStyles,
textStyles
};
5 changes: 5 additions & 0 deletions translations/en.json
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,11 @@
"new_join": "Add observations with the Seek Camera to earn the Challenge badge!",
"inat_admin": "iNat admins: thanks for testing Seek! You're seeing the latest iNat challenge before the start of the month for testing purposes."
},
"updates_card": {
"header": "Seek Updates",
"updated_id_model": "We’ve updated our Identification Model!",
"over_x_species": "You can now identify over 80,000 species in the latest version of Seek!"
},
"post_to_inat_card": {
"post_to_inaturalist": "POST TO INATURALIST",
"uploading_x_observations": {
Expand Down
4 changes: 2 additions & 2 deletions utility/dirStorage.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,12 @@ export const dirHome: string = Platform.select( {
} );

export const dirModel: string = Platform.select( {
ios: `${RNFS.DocumentDirectoryPath}/${modelFiles.IOSMODEL}`,
ios: `${RNFS.MainBundlePath}/${modelFiles.IOSMODEL}`,
android: `${RNFS.DocumentDirectoryPath}/${modelFiles.ANDROIDMODEL}`
} );

export const dirTaxonomy: string = Platform.select( {
ios: `${RNFS.DocumentDirectoryPath}/${modelFiles.IOSTAXONOMY}`,
ios: `${RNFS.MainBundlePath}/${modelFiles.IOSTAXONOMY}`,
android: `${RNFS.DocumentDirectoryPath}/${modelFiles.ANDROIDTAXONOMY}`
} );

Expand Down
Loading

0 comments on commit 6add69f

Please sign in to comment.