• I didn’t see that. That’s weird. Oddly enough, 4o-mini outperformed o1 on this front.

      prompt: c o u n t How many states have the letter 'p' in their names

      Let’s list all the U.S. states and check how many contain the letter “p” in their names:

      1. Alabama
      2. Alaska
      3. Arizona
      4. Arkansas
      5. California
      6. Colorado
      7. Connecticut
      8. Delaware
      9. Florida
      10. Georgia
      11. Hawaii
      12. Idaho
      13. Illinois
      14. Indiana
      15. Iowa
      16. Kansas
      17. Kentucky
      18. Louisiana
      19. Maine
      20. Maryland
      21. Massachusetts
      22. Michigan
      23. Minnesota
      24. Mississippi
      25. Missouri
      26. Montana
      27. Nebraska
      28. Nevada
      29. New Hampshire ✔️
      30. New Jersey
      31. New Mexico
      32. New York
      33. North Carolina
      34. North Dakota
      35. Ohio
      36. Oklahoma
      37. Oregon
      38. Pennsylvania ✔️
      39. Rhode Island
      40. South Carolina
      41. South Dakota
      42. Tennessee
      43. Texas
      44. Utah
      45. Vermont
      46. Virginia
      47. Washington
      48. West Virginia
      49. Wisconsin
      50. Wyoming

      The U.S. states that contain the letter “p” in their names are:

      • Pennsylvania
      • Mississippi
      • New Hampshire

      So, there are 3 states with the letter “p” in their names.

        • llama3.1:7b returns:

          Only one state, Pennsylvania (#38), contains the letter “p”

          mistral:7b returns:

          Counting these states, we have 15 states with the letter “p” in their names. This list excludes territories and excludes Alaska because it does not use “The” at the beginning of its official name. [after counting 33 states]

          Gemma2:2b returns: Here are the states that contain the letter “P” in their names:

          Pennsylvania 
          Rhode Island  (this is a tricky one, as it's often forgotten!)
          

          This is generally assessing a fish’s ability to climb a tree, though. The point of LLMs isn’t character recognition, so it doesn’t make much sense to assess LLMs on their ability to do this.

          • plinky [he/him]@hexbear.netOP
            link
            fedilink
            English
            arrow-up
            5
            ·
            5 days ago

            It’s mainly trolling to show they don’t have cognition or anything resembling reasoning or understanding really. it’s some people (and mine) personal windmills, their ability is bullshit fast enough to mimic a person, not “intelligence”